X2Y 1.1.7 Document Processor Released!
X2Y is a flexible, configurable and extendable server-based document processing framework written in python. It has the following feaures: - Cross platform - Fully documented - Run as either a cron job, scheduled task or by hand - Fully configurable logging and notification - Fully configurable processing chains, including multiple seperate processing chains on a single server - Scriptable conversion plugins, inputs and outputs - Processors can be written in ANY language - Input from local file system, http get or imap folder supplied as standard - Output to local file system, http post or mail message supplied as standard - Supplied processors for EasyPDF and OpenOffice for document conversion Potential applications include server-side document conversion, virus scanning. We are particularly interested in hearing from anyone who develops new inputs, outputs and processors! For more information, please see: http://www.simplistix.co.uk/software/applications/x2y cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
MailingLogger 3.1.0 Released!
Hot on the heals of the 3.0.0 release, this 3.1.0 release offers support for SMTP hosts that require authentication in order to send mail... Mailinglogger enables log entries to be emailed either as the entries are logged or as a summary at the end of the running process. This pair of enhanced emailing handlers for the python logging framework is now available as a standard python package and as an egg. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with an X-Mailer header for easy filtering - flood protection to ensure the number of emails sent is not excessive - fully documented and tested In addition, extra support is provided for configuring the handlers when using ZConfig, Zope 2 or Zope 3. Installation is as easy as: easy_install mailinglogger For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
MailingLogger 3.2.0 Released!
With help from Jens Vagelpohl, I'm pleased to announce a new release of Mailinglogger that now supports filtering of log entries... Mailinglogger enables log entries to be emailed either as the entries are logged or as a summary at the end of the running process. This pair of enhanced emailing handlers for the python logging framework is now available as a standard python package and as an egg. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with an X-Mailer header for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested In addition, extra support is provided for configuring the handlers when using ZConfig, Zope 2 or Zope 3. Installation is as easy as: easy_install mailinglogger For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
MailingLogger 3.2.2 Released!
I'm pleased to announce a new release of Mailinglogger that finally correctly supports easy_install and so works fine with zc.buildout-based projects. In fact, MailingLogger has *become* a zc.buildout-based project for its development... Anyway, Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with an X-Mailer header for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested In addition, extra support is provided for configuring the handlers when using ZConfig, Zope 2 or Zope 3. Installation is as easy as: easy_install mailinglogger For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger or http://pypi.python.org/pypi/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Tutorial on working with Excel files in Python (without COM and cross platform!) at PyConUS 2009
Hi All, Too many people in the Python community think the only way to work with Excel files in Python is using COM on Windows. To try and correct this, I'm giving a tutorial at this year's PyCon in Chicago on Wednesday, 25th March that will cover working with Excel files in Python using the pure-python libraries xlrd, xlwt and xlutils. I'll be looking to cover: - Reading Excel Files Including formatting, unicode dates and formulae. - Writing Excel Files Including formatting with easyxf and things like freeze pains, print areas, etc - Filtering Excel Files A run through on the structure of xlutils.filter and some examples to show you how it works. - Workshop for your problems I'm hoping anyone who attends will get a lot out of this! If you're planning on attending and have a particular problem you'd like to work on in this part of the tutorial, please drop me an email and I'll try and make sure I come prepared! All you need for the tutorial is a working knowledge of Excel and Python, with a laptop as an added benefit, and to be at PyCon this year: http://us.pycon.org I look forward to seeing you all there! Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Announcing www.python-excel.org
Hi All, Google unfortunately has a knack of presenting prospective Python users who need to work with Excel files with information that is now really rather out of date. To try and help with this, I've setup a small website at: http://www.python-excel.org ...to try and list the latest recommended ways of working with Excel files in Python. If you work with excel files in Python, then please take a look and let me know what you think. cheers, Chris PS: If anyone reading this has a python-related blog, it might help Google if you could post a short entry about this new site. -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Tutorial on working with Excel files in Python (without COM and cross platform!) at EuroPython 2009
Hi All, Too many people in the Python community *still* think the only way to work with Excel files in Python is using COM on Windows. To try and correct this, I'm giving a tutorial at this year's EuroPython conference in Birmingham, UK on Monday, 29th June that will cover working with Excel files in Python using the pure-python libraries xlrd, xlwt and xlutils. I'll be looking to cover: - Reading Excel Files Including extracting all the data types, formatting and working with large files. - Writing Excel Files Including formatting, many of the useful frilly extras and writing large excel files. - Modifying and Filtering Excel Files A run through of taking existing Excel files and modifying them in various ways. - Workshop for your problems I'm hoping anyone who attends will get a lot out of this! If you're planning on attending and have a particular problem you'd like to work on in this part of the tutorial, please drop me an email and I'll try and make sure I come prepared! All you need for the tutorial is a working knowledge of Excel and Python, with a laptop as an added benefit, and to be at EuroPython this year: http://www.europython.eu/ I look forward to seeing you all there! Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
xlutils 1.3.2 released!
Hi All, I'm pleased to announce a new release of xlutils. This is a small collection of utilities that make use of both xlrd and xlwt to process Microsoft Excel files. The list of utilities included in this release are: xlutils.copy Tools for copying xlrd.Book objects to xlwt.Workbook objects. xlutils.display Utility functions for displaying information about xlrd-related objects in a user-friendly and safe fashion. xlutils.filter A mini framework for splitting and filtering Excel files into new Excel files. xlutils.margins Tools for finding how much of an Excel file contains useful data. xlutils.save Tools for serializing xlrd.Book objects back to Excel files. xlutils.styles Tools for working with formatting information expressed in styles. A full list of changes since the last release can be found here: http://www.simplistix.co.uk/software/python/xlutils/changes To find out more, please read here: http://www.simplistix.co.uk/software/python/xlutils In case you're not aware, xlrd and xlwt are two excellent pure-python libraries for reading and writing Excel files. They run on any platform and, likely, any implementation of Python without the need for horrific things like binding to Excel via COM and so needing a Windows machine. If you use any of xlrd, xlwt or xlutils, the following google group will be of use: http://groups.google.com.au/group/python-excel Hope some of this is of interest, I'd love to hear from anyone who ends up using it! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
TestFixtures 1.6.0 released!
Hi All, I'm pleased to announce the first advertised release of TestFixtures. This is a collection of helpers and mock objects that are useful when writing unit tests or doc tests. The modules currently included are: *Comparison* This class lets you instantiate placeholders that can be used to compared expected results with actual results where objects in the actual results do not support useful comparison. The comparision can be based just on the type of the object or on a partial set of the object's attributes, both of which are particularly handy when comparing sequences returned from tested code. *compare* A replacement for assertEquals and the failUnless(x() is True) pattern. Gives more useful differences when the arguments aren't the same, particularly for sequences and long strings. *diff* This function will compare two strings and give a unified diff of their comparison. Handy as a third parameter to unittest.TestCase.assertEquals. *generator* This function will return a generator that yields the arguments it was called with when the generator is iterated over. *LogCapture* This helper allows you to capture log messages for specified loggers in doctests. *log_capture* This decorator allows you to capture log messages for specified loggers for the duration of unittest methods. *replace* This decorator enables you to replace objects such as classes and functions for the duration of a unittest method. The replacements are removed regardless of what happens during the test. *Replacer* This helper enables you to replace objects such as classes and functions from within doctests and then restore the originals once testing is completed. *should_raise* This is a better version of assertRaises that lets you check the exception raised is not only of the correct type but also has the correct parameters. *TempDirectory* This helper will create a temporary directory for you using mkdtemp and provides a handy class method for cleaning all of these up. *tempdir* This decorator will create a temporary directory for the duration of the unit test and clear it up no matter the outcome of the test. *test_date* This is a handy class factory that returns datetime.date replacements that have a today method that gives repeatable, specifiable, testable dates. *test_datetime* This is a handy class factory that returns datetime.datetime replacements that have a now method that gives repeatable, specifiable, testable datetimes. *test_time* This is a handy replacement for time.time that gives repeatable, specifiable, testable times. *wrap* This is a generic decorator for wrapping method and function calls with a try-finally and having code executed before the try and as part of the finally. To find out more, please read here: http://pypi.python.org/pypi/testfixtures cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
MailingLogger 3.3.0 Released!
I'm pleased to announce a new release of Mailinglogger.
Mailinglogger provides two handlers for the standard python
logging framework that enable log entries to be emailed either as the
entries are logged or as a summary at the end of the running process.
The handlers have the following features:
- customisable and dynamic subject lines for emails sent
- emails sent with an X-Mailer header for easy filtering
- flood protection to ensure the number of emails sent is not excessive
- support for SMTP servers that require authentication
- fully documented and tested
In addition, extra support is provided for configuring the handlers when
using ZConfig, Zope 2 or Zope 3.
The latest releases of ZConfig, in particular, provide a great way to
configure the python logging framework without having to resort to the
appalling .ini-based configuration stuff:
from ZConfig import configureLoggers
configureLoggers('''
... logger
...level INFO
...logfile
... PATH STDOUT
... format %(levelname)s %(name)s %(message)s
.../logfile
... /logger
... ''')
This release of MailingLogger adds the ability to use %(levelname)s in
subject-line formatting for SummarisingLoggers to include the highest
level logged.
For more information, please see:
http://www.simplistix.co.uk/software/python/mailinglogger
or
http://pypi.python.org/pypi/mailinglogger
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations.html
ErrorHandler 1.0.0 Released!
I'm pleased to finally get around to announcing the release of ErrorHandler.
This is a handler for the python standard logging framework that can
be used to tell whether messages have been logged at or above a
certain level.
This can be useful when wanting to ensure that no errors have been
logged before committing data back to a database.
Here's an example:
First, you set up the error handler:
from logging import getLogger
from errorhandler import ErrorHandler
logger = getLogger()
e = ErrorHandler()
The handler started off being un-fired:
e.fired
False
Then you do whatever else you need to do, which may involve logging:
logger.info('some information')
e.fired
False
However, if any logging occurs at an error level or above:
logger.error('an error')
Then the error handler becomes fired:
e.fired
True
You use this as a condition to only do certain actions when no errors
have been logged:
if e.fired:
... print Not updating files as errors have occurred
Not updating files as errors have occurred
If your code does work in batches, you may wish to reset the error
handler at the start of each batch:
e.reset()
e.fired
False
For more information, please see:
http://www.simplistix.co.uk/software/python/errorhandler
or
http://pypi.python.org/pypi/errorhandler
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations.html
xlutils 1.4.0 released!
Hi All, I'm pleased to announce a new release of xlutils. This is a small collection of utilities that make use of both xlrd and xlwt to process Microsoft Excel files. The changes for this release are as follows: - Add sheet density information and onesheet option to xlutils.margins. - Reduced the memory footprint of xlutils.filter.ColumnTrimmer at the expense of speed. - Fixed incorrect warnings about boolean cells in xlutils.filter.ErrorFilter. xlwt has always supported boolean cells. - xlutils.filter.BaseReader now opens workbooks with on_demand = True - Added support for xlrd Books opened with on_demand as True passed to open_workbook. - Fixed bug when copying error cells. - Requires the latest versions of xlrd (0.7.1) and xlwt (0.7.2). To find out more, please read here: http://www.simplistix.co.uk/software/python/xlutils In case you're not aware, xlrd and xlwt are two excellent pure-python libraries for reading and writing Excel files. They run on any platform and, likely, any implementation of Python without the need for horrific things like binding to Excel via COM and so needing a Windows machine. If you use any of xlrd, xlwt or xlutils, the following google group will be of use: http://groups.google.com.au/group/python-excel Hope some of this is of interest, I'd love to hear from anyone who ends up using it! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
TestFixtures 1.6.1 released!
Hi All, I'm pleased to announce a new release of TestFixtures. This package is a collection of helpers and mock objects that are useful when writing unit tests or doc tests. This release sees the following changes: - @replace and Replacer.replace can now replace attributes that may not be present, provided the `strict` parameter is passed as False. - should_raise now catches BaseException rather than Exception so raising of SystemExit and KeyboardInterrupt can be tested. To find out more, please read here: http://pypi.python.org/pypi/testfixtures cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
xlutils 1.4.1 released!
Hi All, I'm pleased to announce a new release of xlutils. This package is a small collection of utilities that make use of both xlrd and xlwt to process Microsoft Excel files. This release includes memory and speed enhancements for xlutils.filter and xlutils.copy. To find out more, please read here: http://www.simplistix.co.uk/software/python/xlutils In case you're not aware, xlrd and xlwt are two excellent pure-python libraries for reading and writing Excel files. They run on any platform and, likely, any implementation of Python without the need for horrific things like binding to Excel via COM and so needing a Windows machine. If you use any of xlrd, xlwt or xlutils, the following google group will be of use: http://groups.google.com.au/group/python-excel Hope some of this is of interest, I'd love to hear from anyone who ends up using it! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
ErrorHandler 1.1.0 Released!
I'm pleased to announce a new release of ErrorHandler. This is a handler for the python standard logging framework that can be used to tell whether messages have been logged at or above a certain level. The only change for this release is that there is now a full set of documentation available courtesy of Sphinx: http://packages.python.org/errorhandler/ For more information, please see: http://www.simplistix.co.uk/software/python/errorhandler cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Checker 1.0 Released!
I'm pleased to announce the first release of Checker. This is a cross-platform, pluggable tool for comparing the configuration of a machine with a known configuration stored in text files in a source control system all written in Python. For more information, please see: http://www.simplistix.co.uk/software/python/checker cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Checker 1.3 Released!
I'm pleased to announce a new release of Checker. This is a cross-platform, pluggable tool for comparing the configuration of a machine with a known configuration stored in text files in a source control system all written in Python. This release and the previous release fix ordering issues in file listings as well as generalise operating package listing to support RedHat-like systems in addition to Debian-like systems. For more information, please see: http://www.simplistix.co.uk/software/python/checker cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Execute 1.2 Released!
I'm pleased to announce the first public release of Execute. This is a collection of common patterns for executing commands as sub processes. It supports executing a simple command that requires no input in a sub process and can return: - text sent to the standard error and output streams - the return code set by the command - both of the above as a tuple For more information, please see: http://www.simplistix.co.uk/software/python/execute cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Checker 1.4 Released!
I'm pleased to announce a new release of Checker. This is a cross-platform, pluggable tool for comparing the configuration of a machine with a known configuration stored in text files in a source control system all written in Python. This release adds a 'command' checker that lets you record and check the output of arbitrary commands. The initial use case was for dealing with the output of 'chkconfig --list' on Red Hat. For more information, please see: http://www.simplistix.co.uk/software/python/checker cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: MailingLogger 3.4.0 Released!
Heh, of course, I forgot the setuptools-git extension to make include_package_data=True work, so this release was pretty useless, other than the docs on packages.python.org/testfixtures ;-) Anyway, 3.4.1 has now been released which fixes this! cheers, Chris On 17/08/2011 23:37, Chris Withers wrote: I'm pleased to announce a new release of Mailinglogger. Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with a configurable headers for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested This release has no functional changes but finally ships with a full new set of Sphinx docs: http://packages.python.org/mailinglogger/ For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger or http://pypi.python.org/pypi/mailinglogger cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
MailingLogger 3.5.0 Released!
I'm pleased to announce a new release of Mailinglogger. Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with a configurable headers for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested The only change for this release was to add an X-Log-Level header to emails sent. For MailingLogger, this is the level of the log message being emailed. For SummarisingLogger this is the highest level of any of the messages handled. Full docs can be found here: http://packages.python.org/mailinglogger/ For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger or http://pypi.python.org/pypi/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
MailingLogger 3.6.0 Released!
I'm pleased to announce a new release of Mailinglogger. Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with a configurable headers for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested The only change for this release is to allow summaries sent by SummarisingLogger to contain messages logged at a lower level than those which triggered the summary to be emailed. Full docs can be found here: http://packages.python.org/mailinglogger/ For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger or http://pypi.python.org/pypi/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
MailingLogger 3.7.0 Released!
I'm pleased to announce a new release of Mailinglogger. Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with a configurable headers for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested This release fixes a long standing bug that occurred when logging unicode messages. It also provides support for sending colourised HTML emails: http://packages.python.org/mailinglogger/html.html Full docs can be found here: http://packages.python.org/mailinglogger/ For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger or http://pypi.python.org/pypi/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
X2Y 1.1.7 Document Processor Released!
X2Y is a flexible, configurable and extendable server-based document processing framework written in python. It has the following feaures: - Cross platform - Fully documented - Run as either a cron job, scheduled task or by hand - Fully configurable logging and notification - Fully configurable processing chains, including multiple seperate processing chains on a single server - Scriptable conversion plugins, inputs and outputs - Processors can be written in ANY language - Input from local file system, http get or imap folder supplied as standard - Output to local file system, http post or mail message supplied as standard - Supplied processors for EasyPDF and OpenOffice for document conversion Potential applications include server-side document conversion, virus scanning. We are particularly interested in hearing from anyone who develops new inputs, outputs and processors! For more information, please see: http://www.simplistix.co.uk/software/applications/x2y cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] PEP 382: Namespace Packages
Martin v. Löwis wrote: I propose the following PEP for inclusion to Python 3.1. Please comment. Would this support the following case: I have a package called mortar, which defines useful stuff: from mortar import content, ... I now want to distribute large optional chunks separately, but ideally so that the following will will work: from mortar.rbd import ... from mortar.zodb import ... from mortar.wsgi import ... Does the PEP support this? The only way I can currently think to do this would result in: from mortar import content,.. from mortar_rbd import ... from mortar_zodb import ... from mortar_wsgi import ... ...which looks a bit unsightly to me. cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] PEP 382: Namespace Packages
P.J. Eby wrote: Apart from that, this mechanism sounds great! I only wish there was a way to backport it all the way to 2.3 so I could drop the messy bits from setuptools. Maybe we could? :-) Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] PEP 382: Namespace Packages
andrew cooke wrote: I now want to distribute large optional chunks separately, but ideally so that the following will will work: from mortar.rbd import ... from mortar.zodb import ... from mortar.wsgi import ... i may be misunderstanding, but i think you can already do this. in lepl i have code spread across many modules (equivalent to your mortar.rbd, i have lepl.matchers etc). then in lepl/__init__.py i import those and define __this__ to export them into the lepl namespace. so you can import either do: Okay, but do you: - distribute lepl.matchers in a seperate distribution to lepl? - have actual code in the lepl package? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] PEP 382: Namespace Packages
Martin v. Löwis wrote: Chris Withers wrote: Martin v. Löwis wrote: I propose the following PEP for inclusion to Python 3.1. Please comment. Would this support the following case: I have a package called mortar, which defines useful stuff: from mortar import content, ... I now want to distribute large optional chunks separately, but ideally so that the following will will work: from mortar.rbd import ... from mortar.zodb import ... from mortar.wsgi import ... Does the PEP support this? That's the primary purpose of the PEP. Are you sure? Does the pep really allow for: from mortar import content from mortar.rdb import something ...where 'content' is a function defined in mortar/__init__.py and 'something' is a function defined in mortar/rdb/__init__.py *and* the following are separate distributions on PyPI: - mortar - mortar.rdb ...where 'mortar' does not contain 'mortar.rdb'. You can do this today already (see the zope package, No, they have nothing but a (functionally) empty __init__.py in the zope package. cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-Dev] PEP 382: Namespace Packages
P.J. Eby wrote: See the third paragraph of http://www.python.org/dev/peps/pep-0382/#discussion Indeed, I guess the PEP could be made more explanatory then 'cos, as a packager, I don't see what I'd put in the various setup.py and __init__.py to make this work... That said, I'm delighted to hear it's going to be possible and wholeheartedly support the PEP and it's backporting to 2.7 as a result... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
MailingLogger 3.2.0 Released!
With help from Jens Vagelpohl, I'm pleased to announce a new release of Mailinglogger that now supports filtering of log entries... Mailinglogger enables log entries to be emailed either as the entries are logged or as a summary at the end of the running process. This pair of enhanced emailing handlers for the python logging framework is now available as a standard python package and as an egg. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with an X-Mailer header for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested In addition, extra support is provided for configuring the handlers when using ZConfig, Zope 2 or Zope 3. Installation is as easy as: easy_install mailinglogger For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
securing a python execution environment...
Hi All, I'm trying to build a secure execution environment for bits of python for two reasons: - to allow users of the system to write scripts in python without circumventing the application's security model - to allow the system to have an environment where security is handled without having to do explicit checks in every piece of example code. This second point is better demonstrated by an example: Bad: from security import check,AccessDenied if check(someobj,'someattr'): print someattr else: raise AccessDenied(can't access 'someattr') Traceback (most recent call last): File stdin, line ?, in ? AccessDenied: can't access 'someattr' Good: someobj.someattr Traceback (most recent call last): File stdin, line ?, in ? AccessDenied: can't access 'someattr' Now, I think I can get a lot of this from Zope 3's security proxy objects, however I need to find a way to limit the importing of modules to, for example, prevent people importing the method that unwraps the proxy objects ;-) Have other people bumped into this problem? What solutions do people recommend? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: What is python?????
Diez B. Roggisch wrote: It's amazing that recently so many spam is written by some guys actually _reading_ this group. I presume they somehow want to create credibility in their postings by provoking real threads that then at some point contain the actual information. I wonder why this group/list isn't moderated in such a way that posts by new members are moderated at first until they're seen to be human. (and non-spamming humans to boot ;-) ) I admin a few low volume lists and it's proved very close to 100% successful in killing off the spam problem :-) cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: securing a python execution environment...
Laszlo Nagy wrote: Once upon a time, there has been a module called bastillon (am I right?) and rexec (restricted execution environment) but they were not really secure. It was a long time ago. Python is very flexible, and interpreted and it is hard to prevent the users from importing modules. Indeed. I think Zope's security proxies solve a lot of the problem as they are a C extension to python and so can't be circumvented ;-) They can do nasty things. For example, open a file and eval() it etc. Yes, there are plenty of builtins that need to be blocked out and plenty of things that need to be blocked from being imported, but I know it is possible ;-) (see Zope's Script (Python) objects, I'm just hoping for a cleaner, simpler solution...) cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: securing a python execution environment...
Alberto Berti wrote: maybe using import hooks? http://www.python.org/dev/peps/pep-0302/ Well, as Lazlo reminded me, there are also plenty of builtins that are problematic... although hopefully providing a limited set of contents for the global and local namespaces could solve that? But, regardless of those, how do you prevent someone undoing the hooks, regardless of which ones you use? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: securing a python execution environment...
Paul Boddie wrote: http://wiki.python.org/moin/SandboxedPython http://wiki.python.org/moin/How_can_I_run_an_untrusted_Python_script_safely_%28i%2ee%2e_Sandbox%29 Yeah, from this I'm pretty much set on: http://pypi.python.org/pypi/RestrictedPython/ I know it's pretty bulletproof (I've been using it inderectly for years!) but I do worry that it may be a little slow. I guess some poking will tell... If anyone's interested, I'll be continuing this discussion here: http://mail.zope.org/pipermail/zope-dev/2007-November/030368.html (this list is a little too high colume for me :-S) cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
MailingLogger 3.0.0 Released!
Mailinglogger enables log entries to be emailed either as the entries are logged or as a summary at the end of the running process. This pair of enhanced emailing handlers for the python logging framework is now available as a standard python package and as an egg. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with an X-Mailer header for easy filtering - flood protection to ensure the number of emails sent is not excessive - fully documented and tested In addition, extra support is provided for configuring the handlers when using ZConfig, Zope 2 or Zope 3. Installation is as easy as: easy_install mailinglogger For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
MailingLogger 3.1.0 Released!
Hot on the heals of the 3.0.0 release, this 3.1.0 release offers support for SMTP hosts that require authentication in order to send mail... Mailinglogger enables log entries to be emailed either as the entries are logged or as a summary at the end of the running process. This pair of enhanced emailing handlers for the python logging framework is now available as a standard python package and as an egg. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with an X-Mailer header for easy filtering - flood protection to ensure the number of emails sent is not excessive - fully documented and tested In addition, extra support is provided for configuring the handlers when using ZConfig, Zope 2 or Zope 3. Installation is as easy as: easy_install mailinglogger For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Problems with email.Generator.Generator
Hi All,
The following piece of code is giving me issues:
from email.Charset import Charset,QP
from email.MIMEText import MIMEText
charset = Charset('utf-8')
charset.body_encoding = QP
msg = MIMEText(
u'Some text with chars that need encoding: \xa3',
'plain',
)
msg.set_charset(charset)
print msg.as_string()
Under Python 2.4.2, this produces the following output, as I'd expect:
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=utf-8
Some text with chars that need encoding: =A3
However, under Python 2.4.3, I now get:
Traceback (most recent call last):
File test_encoding.py, line 14, in ?
msg.as_string()
File c:\python24\lib\email\Message.py, line 129,
in
as_string
g.flatten(self, unixfrom=unixfrom)
File c:\python24\lib\email\Generator.py, line 82,
in flatten
self._write(msg)
File c:\python24\lib\email\Generator.py, line 113,
in _write
self._dispatch(msg)
File c:\python24\lib\email\Generator.py, line 139,
in
_dispatch
meth(msg)
File c:\python24\lib\email\Generator.py, line 182,
in
_handle_text
self._fp.write(payload)
UnicodeEncodeError: 'ascii' codec can't encode
character
u'\xa3' in position 41:
ordinal not in range(128)
This seems to be as a result of this change:
http://svn.python.org/view/python/branches/release24-maint/Lib/email/Generator.py?rev=42272r1=37910r2=42272
...which is referred to as part of a fix for this bug:
http://sourceforge.net/tracker/?func=detailaid=1409455group_id=5470atid=105470
Now, is this change to Generator.py in error or am I doing something wrong?
If the latter, how can I change my code such that it works as I'd expect?
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Manlio Perillo wrote: Try with: msg = MIMEText( u'Some text with chars that need encoding: \xa3', _charset='utf-8', ) and you will obtain the error: Traceback (most recent call last): File pyshell#4, line 3, in -toplevel- _charset='utf-8', File C:\Python2.4\lib\email\MIMEText.py, line 28, in __init__ self.set_payload(_text, _charset) File C:\Python2.4\lib\email\Message.py, line 218, in set_payload self.set_charset(charset) File C:\Python2.4\lib\email\Message.py, line 260, in set_charset self._payload = charset.body_encode(self._payload) File C:\Python2.4\lib\email\Charset.py, line 366, in body_encode return email.base64MIME.body_encode(s) File C:\Python2.4\lib\email\base64MIME.py, line 136, in encode enc = b2a_base64(s[i:i + max_unencoded]) UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in position 41: ordinal not in range(128) OK, but I fail to see how replacing one unicode error with another is any help... :-S Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Peter Otten wrote: http://sourceforge.net/tracker/?func=detailaid=1409455group_id=5470atid=105470 Now, is this change to Generator.py in error or am I doing something wrong? I'm not familiar enough with the email package to answer that. I'm hoping someone around here is ;-) If the latter, how can I change my code such that it works as I'd expect? email.Generator and email.Message use cStringIO.StringIO internally, which can't cope with unicode. A quick fix might be to monkey-patch: I'm not sure that's correct, but I'm happy to stand corrected. My understanding is that the StringIO's don't mind as long as they type is consistent - ie: con't mix unicode and encoded strings, 'cos it forced python's default ascii codec to kick in and spew unicode errors. Now, I want to know what I'm supposed to do when I have unicode source and want it to end up as either a text/plain or text/html mime part. Is there a how-to for this anywhere? The email package's docs are short on examples involving charsets, unicode and the like :-( Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Steve Holden wrote: Is there a how-to for this anywhere? The email package's docs are short on examples involving charsets, unicode and the like :-( Well, it would seem like the easiest approach is to monkey-patch the use of cStringIO to StringIO as recommended and see if that fixes your problem. Wouldn't it? No, not really, since at best that's a nasty (and I meant really nasty) hack. I'm using the email package as part of a library that I'm building which is to be used with various frameworks. Monkey patching modules is about as bad as it gets in that situation... At worst, and most likely based on my past experience of (c)StringIO being used to accumulate output, it won't make a jot of difference... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Manlio Perillo wrote: The problem is simple: email package does not support well Unicode strings. Really? All the character set support seems to indicate a fair bit of thought went into this aspect, although it does appear that no-one bothered to document it :-( Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Peter Otten wrote: What past experience? StringIO.StringIO().write(unichr(128)) cStringIO.StringIO().write(unichr(128)) Traceback (most recent call last): File stdin, line 1, in ? UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position 0: ordinal not in range(128) OK, I stand corrected, although I suspect the bug is actually in StringIO.StringIO in that it doesn't barf on unicodes. (Python 3000 and all that) Which again leads us back to the email package: it used to do the right thing from what I can see, and now it doesn't, and ends up trying to write a unicode to a cStringIO, which (rightly, I guess) barfs... Barry, Barry, where are you? ;-) Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Peter Otten wrote:
Chris Withers wrote:
At worst, and most likely based on my past experience of (c)StringIO
being used to accumulate output, it won't make a jot of difference...
What past experience?
StringIO.StringIO().write(unichr(128))
cStringIO.StringIO().write(unichr(128))
Traceback (most recent call last):
File stdin, line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\x80' in position
0: ordinal not in range(128)
Okay, more out of desperation than anything else, lets try this:
from email.Charset import Charset,QP
from email.MIMEText import MIMEText
from StringIO import StringIO
from email import Generator,Message
Generator.StringIO = Message.StringIO = StringIO
charset = Charset('utf-8')
charset.body_encoding = QP
msg = MIMEText(u'Some text with chars that need encoding: \xa3','plain')
msg.set_charset(charset)
print repr(msg.as_string())
u'MIME-Version: 1.0\nContent-Transfer-Encoding: 8bit\nContent-Type:
text/plain; charset=utf-8\n\nSome text with chars that need encoding:
\xa3'
Yay! No unicode error, but also no use:
File c:\python24\lib\smtplib.py, line 692, in sendmail
(code,resp) = self.data(msg)
File c:\python24\lib\smtplib.py, line 489, in data
self.send(q)
File c:\python24\lib\smtplib.py, line 316, in send
self.sock.sendall(str)
File string, line 1, in sendall
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in
position 297: ordinal not in range(128)
The other variant I've tried is:
from email.Charset import Charset,QP
from email.MIMEText import MIMEText
charset = Charset('utf-8')
charset.body_encoding = QP
msg = MIMEText('','plain',)
msg.set_charset(charset)
msg.set_payload(charset.body_encode(u'Some text with chars that need
encoding: \xa3'))
print msg.as_string()
Which is sort of okay:
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=utf-8
Some text with chars that need encoding: =A3
...except it gets the transfer encoding wrong, which means Thunderbird
shows =A3 instead of the pound sign that it should :-(
...this is down to a pretty lame bit of code in Encoders.py which
basically checks for a unicode error *sigh*
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Chris Withers wrote:
...except it gets the transfer encoding wrong, which means Thunderbird
shows =A3 instead of the pound sign that it should :-(
...this is down to a pretty lame bit of code in Encoders.py which
basically checks for a unicode error *sigh*
OK, slight progress... here a new version that actually works:
from email.Charset import Charset,QP
from email.MIMEText import MIMEText
charset = Charset('utf-8')
charset.body_encoding = QP
msg = MIMEText('','plain',None)
msg.set_payload(u'Some text with chars that need encoding:\xa3',charset)
print msg.as_string()
MIME-Version: 1.0
Content-Type: text/plain; charset; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Some text with chars that need encoding:=A3
Okay, so this actually does the right thing... wahey!
...but hold your horses, if Charset isn't set to quoted printable, then
you end up with problems:
charset = Charset('utf-8')
msg = MIMEText('','plain',None)
msg.set_payload(u'Some text with chars that need encoding:\xa3',charset)
Traceback (most recent call last):
File C:\test_encoding.py, line 5, in ?
msg.set_payload(u'Some text with chars that need
encoding:\xa3',charset)
File c:\python24\lib\email\Message.py, line 218, in set_payload
self.set_charset(charset)
File c:\python24\lib\email\Message.py, line 260, in set_charset
self._payload = charset.body_encode(self._payload)
File c:\python24\lib\email\Charset.py, line 366, in body_encode
return email.base64MIME.body_encode(s)
File c:\python24\lib\email\base64MIME.py, line 136, in encode
enc = b2a_base64(s[i:i + max_unencoded])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in
position 40: ordinal not in range(128)
Now what?
*sigh*
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Chris Withers wrote:
print msg.as_string()
MIME-Version: 1.0
Content-Type: text/plain; charset; charset=utf-8
^^^
Actually, even this isn't correct as you can see above...
charset = Charset('utf-8')
msg = MIMEText('','plain',None)
msg.set_payload(u'Some text with chars that need encoding:\xa3',charset)
Traceback (most recent call last):
File C:\test_encoding.py, line 5, in ?
msg.set_payload(u'Some text with chars that need
encoding:\xa3',charset)
File c:\python24\lib\email\Message.py, line 218, in set_payload
self.set_charset(charset)
File c:\python24\lib\email\Message.py, line 260, in set_charset
self._payload = charset.body_encode(self._payload)
File c:\python24\lib\email\Charset.py, line 366, in body_encode
return email.base64MIME.body_encode(s)
File c:\python24\lib\email\base64MIME.py, line 136, in encode
enc = b2a_base64(s[i:i + max_unencoded])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in
position 40: ordinal not in range(128)
...and I'm still left with this problem...
Has no-one ever successfully generated a correctly formatted email with
email.MIMEText where the message includes non-ascii characters?!
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Problems with email.Generator.Generator - Solved?
Chris Withers wrote:
Has no-one ever successfully generated a correctly formatted email with
email.MIMEText where the message includes non-ascii characters?!
I'm guessing not ;-)
Well, I think I have a winner, but it required me to subclass MIMEText:
from email.Charset import Charset,QP
from email.MIMEText import MIMEText as OriginalMIMEText
from email.MIMENonMultipart import MIMENonMultipart
class MIMEText(OriginalMIMEText):
def __init__(self, _text, _subtype='plain', _charset='us-ascii'):
if isinstance(_charset,Charset):
cs = _charset.input_charset
else:
cs = _charset
if isinstance(_text,unicode):
_text = _text.encode(charset.input_charset)
MIMENonMultipart.__init__(self, 'text', _subtype,
**{'charset': cs})
self.set_payload(_text, _charset)
charset = Charset('utf-8')
charset.body_encoding = QP
txt = u'Some text with chars that need encoding:\xa3'
msg = MIMEText(txt,'plain',charset)
print msg.as_string()
Which gives:
Content-Type: text/plain; charset=utf-8
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Some text with chars that need encoding:=C2=A3
It also works with non-QP charsets.
The reason the subclass is needed is because the
MIMNonMultipart.__init__ cannot handle a charset which isn't a simple
string. Since it's needed for that reason, it seems like the right place
to encode any incoming unicode.
So, by my count, there are two bugs:
1. email.MIMEText.MIMEText can't take a real Charset object to its
__init__ method.
2. email.Message.Message.set_payload has no clue about unicode.
Does that sounds fair? If so, should I open SF issues for them?
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: Problems with email.Generator.Generator
Max M wrote:
From the docs:
The payload is either a string in the case of simple message objects or
a list of Message objects for MIME container documents (e.g. multipart/*
and message/rfc822)
Where'd you find that? I must have missed it in my digging :-S
Message objects are always encoded strings. I don't remember seeing that
it should be possible to use a unicode string as a message.
Yes, I guess I just find that surprising in today's everything should
be unicode world.
The charset passed in set_payload(pl ,charset) is the charset the the
string *is* encoded in. Not the charset it *should* be encoded in.
Indeed, although there's still the bug that while set_payload can accept
a Charset instance for its _charset parameter, the __init__ method for
MIMENonMultipart cannot.
Incidentally, here's the class I finally ended up with:
from email.Charset import Charset
from email.MIMEText import MIMEText as OriginalMIMEText
from email.MIMENonMultipart import MIMENonMultipart
class MTText(OriginalMIMEText):
def __init__(self, _text, _subtype='plain', _charset='us-ascii'):
if not isinstance(_charset,Charset):
_charset = Charset(_charset)
if isinstance(_text,unicode):
_text = _text.encode(_charset.input_charset)
MIMENonMultipart.__init__(self, 'text', _subtype,
**{'charset': _charset.input_charset})
self.set_payload(_text, _charset)
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
A new text templating system for python!
Hi All,
I'm proud to announce that after almost a year's development, the first
public release of Twiddler is available!
Twiddler is a simple but flexible templating system for dynamically
generating textual output.
The key features are:
- No need to learn a templating language.
- As simple as possible while still catering for all possible templating
operations.
- Configurable input parsing so source can be html, xml, plain text or
anything you write an input parser for.
- Configurable output rendering so output can be rendered to a unicode
string, an email, or anything you write an output renderer for.
You can find out more and download from here:
http://www.simplistix.co.uk/software/python/twiddler
If you wish to ask questions, pass judgement or get help, please do so
in the following mailing list:
http://groups.google.com/group/twiddler
This is the first public release, so I'm looking for help with the
following:
- Finding anything that should be possible in a templating system but
isn't with Twiddler
- Packaging it up for easier distribution. I know nothing about eggs,
distutils, etc, and so could do with help to package Twiddler so that
it can be more easily installed.
Finally, here's a couple of quick examples:
from twiddler import Twiddler
t = Twiddler('''body
... div id=titleA title/div
... select name=items
... option id=itemAn item/option
... /select
... /body''')
t['title'].replace('My title')
r = t['item'].repeater()
for i in range(1,4):
... r.repeat('Item '+str(i),value=i,name=False)
print t.render()
body
div id=titleMy title/div
select name=items
option id=item value=1Item 1/option
option id=item value=2Item 2/option
option id=item value=3Item 3/option
/select
/body
from twiddler.input.plaintext import PlainText
t = Twiddler('''To: $to
... itemAn item/item
... ''',input=PlainText)
t['to'].replace('John Smith')
r = t['item'].repeater()
for i in range(1,4):
... r.repeat('Item '+str(i),value=i,name=False)
print t.render()
To: John Smith
Item 1
Item 2
Item 3
Thanks for reading this far, I hope you enjoy it!
cheers,
Chris
--
Simplistix - Content Management, Zope Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
xlutils 1.0.0 released!
Hi All, I'm pleased to announce the first release of xlutils. This is a small collection of utilities that make use of both xlrd and xlwt to process Microsoft Excel files. The current utilities included are: xlutils.margins Tools for finding how much of an Excel file contains useful data. xlutils.filter A mini framework for splitting and filtering Excel files into new Excel files. To find out more, please read here: http://www.simplistix.co.uk/software/python/xlutils In case you're not aware, xlrd and xlwt are two excellent pure-python libraries for reading and writing Excel files. They run on any platform and, likely, any implementation of Python without the need for horrific things like binding to Excel via COM and so needing a Windows machine. If you use any of xlrd, xlwt or xlutils, the following google group will be of use: http://groups.google.com.au/group/python-excel Hope some of this is of interest, I'd love to hear from anyone who ends up using it! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
xlutils 1.1.0 released!
Hi All, I'm pleased to announce a new release of xlutils. This is a small collection of utilities that make use of both xlrd and xlwt to process Microsoft Excel files. The list of utilities included in this release are: xlutils.display Utility functions for displaying information about xlrd-related objects in a user-friendly and safe fashion. xlutils.filter A mini framework for splitting and filtering Excel files into new Excel files. xlutils.margins Tools for finding how much of an Excel file contains useful data. xlutils.save Tools for serializing xlrd.Book objects back to Excel files. xlutils.styles Tools for working with formatting information expressed in styles. To find out more, please read here: http://www.simplistix.co.uk/software/python/xlutils In case you're not aware, xlrd and xlwt are two excellent pure-python libraries for reading and writing Excel files. They run on any platform and, likely, any implementation of Python without the need for horrific things like binding to Excel via COM and so needing a Windows machine. If you use any of xlrd, xlwt or xlutils, the following google group will be of use: http://groups.google.com.au/group/python-excel Hope some of this is of interest, I'd love to hear from anyone who ends up using it! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
MailingLogger 3.2.2 Released!
I'm pleased to announce a new release of Mailinglogger that finally correctly supports easy_install and so works fine with zc.buildout-based projects. In fact, MailingLogger has *become* a zc.buildout-based project for its development... Anyway, Mailinglogger provides two handlers for the standard python logging framework that enable log entries to be emailed either as the entries are logged or as a summary at the end of the running process. The handlers have the following features: - customisable and dynamic subject lines for emails sent - emails sent with an X-Mailer header for easy filtering - flood protection to ensure the number of emails sent is not excessive - support for SMTP servers that require authentication - fully documented and tested In addition, extra support is provided for configuring the handlers when using ZConfig, Zope 2 or Zope 3. Installation is as easy as: easy_install mailinglogger For more information, please see: http://www.simplistix.co.uk/software/python/mailinglogger or http://pypi.python.org/pypi/mailinglogger cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Tutorial on working with Excel files in Python (without COM and cross platform!) at PyConUS 2009
Hi All, Too many people in the Python community think the only way to work with Excel files in Python is using COM on Windows. To try and correct this, I'm giving a tutorial at this year's PyCon in Chicago on Wednesday, 25th March that will cover working with Excel files in Python using the pure-python libraries xlrd, xlwt and xlutils. I'll be looking to cover: - Reading Excel Files Including formatting, unicode dates and formulae. - Writing Excel Files Including formatting with easyxf and things like freeze pains, print areas, etc - Filtering Excel Files A run through on the structure of xlutils.filter and some examples to show you how it works. - Workshop for your problems I'm hoping anyone who attends will get a lot out of this! If you're planning on attending and have a particular problem you'd like to work on in this part of the tutorial, please drop me an email and I'll try and make sure I come prepared! All you need for the tutorial is a working knowledge of Excel and Python, with a laptop as an added benefit, and to be at PyCon this year: http://us.pycon.org I look forward to seeing you all there! Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
ErrorHandler 1.1.0 Released!
I'm pleased to announce a new release of ErrorHandler. This is a handler for the python standard logging framework that can be used to tell whether messages have been logged at or above a certain level. The only change for this release is that there is now a full set of documentation available courtesy of Sphinx: http://packages.python.org/errorhandler/ For more information, please see: http://www.simplistix.co.uk/software/python/errorhandler cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Vote on PyPI comments
Hi All, Apologies for the cross post, but I'm not sure this has received the publicity it deserves... PyPI grew a commenting and rating system a while back, apparently in response to requests from users. However, since it's been rolled out, there's been a backlash from package maintainers who already have mailing lists, bug trackers, etc for their packages and don't want to have to try and keep track of yet another support forum. The arguments for and against are listed here: http://wiki.python.org/moin/PyPIComments To resolve the future of the commenting and rating system, a vote has been set up so everyone can have their say. To vote, please log in to: http://pypi.python.org/pypi ...and follow the instructions you'll be presented with. I would like to remain neutral on this, and for me that means giving package authors the ability to choose whether they want to receive comments, ratings or neither rather than either forcing package authors to accept comments and ratings or abandoning the idea of comments and ratings completely. The closest option to that is: Allow package owners to disallow comments (ratings unmodified) I hope the majority of you feel the same way... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: feedback on function introspection in argparse
Yuv wrote: On Nov 8, 1:33 am, Carl Banks pavlovevide...@gmail.com wrote: Is the docstring expected to be formatted according to some convention? We tried to comply to PEP 257 and we're open to suggestions on this. I'd suggest at the very least supporting Sphinx docstrings that have the parameters in them... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Anything better than shutil?
Roy Smith wrote: The idea interface I see would be one like: shutil.copy([source_dir, '*.conf'], conf_dir) the idea is that if the first argument is a list (or maybe any iterable other than a string?), it would automatically get run through os.path.join(). And, the result would always get passed through glob (), just like a normal shell would. Does anything like this exist? Why don't you wrap up your earlier code: configs = glob.glob(os.path.join(source_dir, '*.conf')) for conf_file in configs: shutil.copy(conf_file, conf_dir) ...in a module with the exact interface you're after? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling Python functions from Excel
Cannonbiker wrote: Hi, unfortunately is my question about server COM (win32com) http://groups.google.com/group/comp.lang.python/browse_thread/thread/ee804cec7f58c6a7# without answer. Please I need Calling Python functions from Excel and receive result back in Excel. Can me somebody advise simplest solution please? I am more VBA programmer than Python. Try http://code.google.com/p/pyinex/ cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Calling Python functions from Excel
Mark Tolonen wrote: Please I need Calling Python functions from Excel and receive result back in Excel. Can me somebody advise simplest solution please? I am more VBA programmer than Python. Try http://code.google.com/p/pyinex/ The book Python: Programming on Win32 has a whole chapter on COM, and a section on COM servers. ...and it's generally accepted that COM sucks rocks through straws, so explore alternatives when they're available ;-) Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Python OpenOffice Spreadsheets
Gerhard Häring wrote: Is there a *simple* way to read OpenOffice spreadsheets? Ironically, if you don't mind working in .xls, which OpenOffice handles just fine, you have xlrd and xlwt to do exactly what you're after: http://www.python-excel.org/ cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
subprocess.Popen and ordering writes to stdout and stderr
Hi All,
I have this simple function:
def execute(command):
process = Popen(command.split(),stderr=STDOUT,stdout=PIPE)
return process.communicate()[0]
..but my unit test for it fails:
from testfixtures import tempdir,compare
from unittest import TestCase
class TestExecute(TestCase):
@tempdir()
def test_out_and_err(self,d):
path = d.write('test.py','\n'.join((
import sys,
sys.stdout.write('stdout\\n'),
sys.stderr.write('stderr\\n'),
sys.stdout.write('stdout2\\n'),
)),path=True)
compare('stdout\nstderr\nstdout2\n',
execute(sys.executable+' '+path))
...because:
AssertionError:
@@ -1,4 +1,4 @@
-stdout
-stderr
-stdout2
+stdout
+stdout2
+stderr
...the order of the writes isn't preserved.
How can I get this to be the case?
Chris
--
Simplistix - Content Management, Batch Processing Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: subprocess.Popen and ordering writes to stdout and stderr
exar...@twistedmatrix.com wrote: How can I get this to be the case? You probably just need to flush stdout and stderr after each write. You set them up to go to the same underlying file descriptor, but they still each have independent buffering on top of that. Okay, but if I do: os.system(sys.executable+' '+path) ...with test.py as-is, I get things in the correct order. How come? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess.Popen and ordering writes to stdout and stderr
exar...@twistedmatrix.com wrote: libc is probably giving you line buffering when you use os.system (because the child process inherits the parent's stdio, and the parent's stdio is probably a pty, and that's the policy libc implements). snip Interesting, but do these assertions still hold true when I tell you that I'm doing all this on Windows? ;-) Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Checker 1.0 Released!
I'm pleased to announce the first release of Checker. This is a cross-platform, pluggable tool for comparing the configuration of a machine with a known configuration stored in text files in a source control system all written in Python. For more information, please see: http://www.simplistix.co.uk/software/python/checker cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: zope server on ubuntu 9.10
S.Selvam wrote:
I am using Ubuntu 9.10 and when i tried to install Zope application
server with the following ,
sudo easy_install -i http://download.zope.org/Zope2/index/2.12.1 Zope2
i got an error as shown below,
---
.
.
Reading http://download.zope.org/Zope2/index/2.12.1/
No local packages or download links found for zope.browsermenu
error: Could not find suitable distribution for
Requirement.parse('zope.browsermenu')
--
I would suggest trying the 2.12.2 that has just been released.
Have you installed any other zope packages or zope-related packages
prior to this?
I'd suggest using the buildout-instance installation instructions here:
http://docs.zope.org/zope2/releases/2.12/INSTALL.html#buildout-instances
Finally, the best list to ask about this stuff is z...@zope.org.
cheers,
Chris
--
Simplistix - Content Management, Batch Processing Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: parsing an Excel formula with the re module
John Machin wrote: The xlwt package (of which I am the maintainer) has a lexer and parser for a largish subset of the syntax ... see http://pypi.python.org/pypi/xlwt xlrd, no? Also worth pointing out that the topic of Python and Excel has its own web site: http://www.python-excel.org ...which links to a more specialist group. Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Append to an Excel file
Niels L. Ellegaard wrote: pp parul.pande...@gmail.com writes: On Jan 9, 1:47 am, Jason Scheirer jason.schei...@gmail.com wrote: On Jan 9, 12:30 am, pp parul.pande...@gmail.com wrote: Hi All, How do I add a line to an existing file. This should append to the existing data in the excel file, which was saved previously. Thanks, PP http://pypi.python.org/pypi/xlwt Hi Jason and all, Thanks I have seen this.. my question is there a way to append to a excel file which has been closed. Any specific modes which can be added to the sheet so that it adds a line to the data which was return in some earlier running of the program. I may be wrong, but I think that you have to do the following 1) Use xlrd to read the file. This creates an xlrd.Book 2) Use xlutils to transform the xlrd.Book into a xlwt.WorkBook 3) Edit the xlwt.WorkBook 4) Save the xlwt.WorkBook https://secure.simplistix.co.uk/svn/xlutils/trunk/xlutils/docs/copy.txt Yup, that's the one... Oh to have funded time to turn all the docs for xl(rd|wt|utils) into nice Sphinx docs... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: unittest inconsistent
Phlip wrote: The reason the 'Tester' object has no attribute 'arg1' is because self still refers to the object made for testA. I hope someone else can spot the low-level reason... ...but why aren't you using http://pypi.python.org/pypi/mock/ ? Look up its patch_object facility... Indeed, I love mock, although I prefer testfixture replace decorator and/or context manager for installing and removing them: http://packages.python.org/testfixtures/mocking.html cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Bugs in CPython 3.1.1 [wave.py]
Alf P. Steinbach wrote: PS: It would be nice if someone(TM) could describe here in detail how to properly report errors like this. http://tinyurl.com/yemcdy7 Of course I'm not going to do it if it involves establishing Yet Another Account somewhere. Of course that means no-one else will know about the problem, and no-one who can fix this problem will find out about it... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: parsing an Excel formula with the re module
John Machin wrote: The OP was planning to dig the formula text out using COM then parse the formula text looking for cell references and appeared to have a rather simplistic view of the ease of parsing Excel formula text -- that's why I pointed him at those facilities (existing, released, proven in the field) in xlwt. Which bits of xlwt are you referring to? (at a guess, the stuff that turns a piece of text into the correct formulae gubbinz when you write a formula to a cell with xlwt?) cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
what test runner should I use?
Hi All, I'm wondering what test runner I should use. Here's my list of requirements: - cross platform (I develop for and on Windows, Linux and Mac) - should not prevent tests from running with other test runners (so no plugins/layers/etc that only work with one specific test runner) - should work with zc.buildout (preferably without a specialist recipe!) So far I've tried the following with the resultant problems: zope.testing - requires a special recipe to be useful - now emits deprecation warnings from itself: https://mail.zope.org/pipermail/zope-dev/2009-December/038965.html - coverage support is baroque to put it politely twisted's trial - only has old-style script definition in setup.py, so doesn't work with buildout without hackery - drops _twisted_trial folders all over the place and doesn't clear them up nose - can't see to get it to run only my packages tests, rather than including the tests of packages my package depends on - seems to be focused towards files rather than modules (which makes it not play nicely with buildout) - seems to be difficult to provide options to at configuration time that can then be overridden on the command line I did also look at py.test's homepage but found it pretty scary. What other options do people recommend? Failing that, any ideas how to fix the problems above? cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: How to use python to register a service (an existing .exe file)
Aahz wrote: In article 4b79e28c$0$4610$426a7...@news.free.fr, News123 news...@free.fr wrote: Is there a python way to register new windows services. I am aware of the instsrv.exe program, which can be used to install services. I could use subprocess.Popen to call instsrv.exe service_name program.exe but wondered, whether there's already an existing function. Use the win32 package. Indeed, there's some intersting code in this area here: http://svn.zope.org/Zope/trunk/src/nt_svcutils/service.py?rev=104819view=auto http://svn.zope.org/Zope/trunk/src/Zope2/Startup/zopectl.py?rev=105396view=auto ...particularly in the do_install function in the latter... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Anything like Effective Java for Python?
kj wrote: Subject line pretty much says it all: is there a book like Effective Java oxymoronic, no? Sorry, couldn't resist ;-) Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.6 still not giving memory back to the OS...
Martin v. Löwis wrote: Today, there are two cases when malloc returns memory on a typical Unix system (in particular, in Linux malloc): a) if the malloc block block is small (below page size), it is allocated from the brk heap, where it can only be returned if the last page of that heap is completely free, and b) if the block is large (multiple pages), it gets returned to the system right away. Case b) can only happen if the C malloc uses mmap to allocate large blocks. I believe (and John will correct me if I'm wrong) that xlrd uses mmap, so why am I not seeing the memory being returned? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Barcodes
Ronn Ross wrote: I found this library, but no documentation(https://cybernetics.hudora.biz/projects/wiki/huBarcode). Has anyone used this or know of a similar library with better documentation? Yup, reportlab has really good barcode generation stuff. I use it to generate labels with barcodes on for asset tracking :-) http://the-gay-bar.com/index.php?/archives/221-Howto-generate-barcodes-in-Python-with-reportlab/ Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
inconsistent sitecustomize.py importing...
Hi All, Give this sitecustomize.py: print ping ...and this script: #!/usr/local/bin/python2.5 print pong! ...both in ~/folder, I would expect the output from: ~$./folder/script ...to be: ping pong ...but sitecustomize.py is not imported :-( If I do: ~$/usr/local/bin/python2.5 folder/script ...still no import. However, if I do: ~/folder$/usr/local/bin/python2.5 script ...sitecustomize.py IS imported! What gives?! Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Even more inconsistent sitecustomize.py importing with PYTHONPATH rolled in...
Chris Withers wrote: However, if I do: ~/folder$/usr/local/bin/python2.5 script ...sitecustomize.py IS imported! However, the following doesn't import sitecustomize.py: ~/folder$ ./script While the following DOES import sitecustomize.py: ~/folder$ export PYTHONPATH= ~/folder$ ./script Bizarrely, none of the following import sitecustomize.py: ~$ export PYTHONPATH=~/folder ~$ ./script ~$ export PYTHONPATH=~/folder/ ~$ ./script ~$ export PYTHONPATH=~/folder ~/folder$ ./script ~$ export PYTHONPATH=~/folder/ ~/folder$ ./script Does anyone have any clue why there's such a huge level of inconsistency here?! Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Even more inconsistent sitecustomize.py importing with PYTHONPATH rolled in...
Chris Withers wrote: Bizarrely, none of the following import sitecustomize.py: ~$ export PYTHONPATH=~/folder ~$ ./script ~$ export PYTHONPATH=~/folder/ ~$ ./script ~$ export PYTHONPATH=~/folder ~/folder$ ./script ~$ export PYTHONPATH=~/folder/ ~/folder$ ./script Okay, brain fail on my part, these do work, the others are still as described though... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: why python got less developers ?
Esam Qanadeely wrote: .NET= i meant all .NET languages What? You mean like Python? ;-) Google IronPython ya troll... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: An iteration idiom (Was: Re: [Guppy-pe-list] loading files containing multiple dumps)
Raymond Hettinger wrote:
In the first case, you would write:
sets.extend(h.load(f))
yes, what I had was:
for s in iter(h.load(f)): sets.append(s)
...which I mistakenly thought was working, but in in fact boils down to
Raymond's code.
The problem is that each item that h.load(f) returns *is* actually an
iterable, so either of the above just ends up the contents of each set
being extended onto `sets` rather than the sets themselved.
It's all really rather confusing, apologies if there's interspersed rant
in here:
from guppy import hpy
h = hpy()
Minor rant, why do I have to instantiate a
class 'guppy.heapy.Use._GLUECLAMP_'
to do anything with heapy?
Why doesn't heapy just expose load, dump, etc?
(oh, and reading the code for guppy.heapy.Use and its ilk made me go
temporarily blind!) ;-)
f = open('copy.hpy')
s = h.load(f)
Less minor rant: this applies to most things to do with heapy... Having
__repr__ return the same as __str__ and having that be a long lump of
text is rather annoying. If you really must, make __str__ return the big
lump of text but have __repr__ return a simple, short, item containing
the class, the id, and maybe the number of contained objects...
Anyway...
id(s)
13905272
len(s)
192
s.__class__
class guppy.heapy.Part.Stat at 0x00CD6A20
i = s[0]
id(i)
13904112
len(i)
1
i.__class__
class guppy.heapy.Part.Stat at 0x00CD6A20
Hmmm, I'm sure there's a good reason why an item in a set has the exact
same class and iterface as a whole set?
It feels like some kind of filtering, where are the docs that explain
all this?
cheers,
Chris
--
Simplistix - Content Management, Batch Processing Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
TestFixtures 1.6.1 released!
Hi All, I'm pleased to announce a new release of TestFixtures. This package is a collection of helpers and mock objects that are useful when writing unit tests or doc tests. This release sees the following changes: - @replace and Replacer.replace can now replace attributes that may not be present, provided the `strict` parameter is passed as False. - should_raise now catches BaseException rather than Exception so raising of SystemExit and KeyboardInterrupt can be tested. To find out more, please read here: http://pypi.python.org/pypi/testfixtures cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
xlutils 1.4.1 released!
Hi All, I'm pleased to announce a new release of xlutils. This package is a small collection of utilities that make use of both xlrd and xlwt to process Microsoft Excel files. This release includes memory and speed enhancements for xlutils.filter and xlutils.copy. To find out more, please read here: http://www.simplistix.co.uk/software/python/xlutils In case you're not aware, xlrd and xlwt are two excellent pure-python libraries for reading and writing Excel files. They run on any platform and, likely, any implementation of Python without the need for horrific things like binding to Excel via COM and so needing a Windows machine. If you use any of xlrd, xlwt or xlutils, the following google group will be of use: http://groups.google.com.au/group/python-excel Hope some of this is of interest, I'd love to hear from anyone who ends up using it! cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: zip a huge file into multiple small ones
krishna chaitanya wrote: I am new to dealing with zip files in python. I have a huge file which i need to zip and send as an attachment through email. My email restrictions are not allowing me to send it in one go. Is there a way to split this file into multiple zip files, so that i can mail them separately. All the individual zip files should be linked. I should be able to extract the original file by extracting the first of the small zip files. You're using the wrong medium. Upload the file to an http server somewhere and just include a link in the email... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: httplib incredibly slow :-(
Dieter Maurer wrote: Chris Withers ch...@simplistix.co.uk writes on Thu, 13 Aug 2009 08:20:37 +0100: ... I've already established that the file downloads in seconds with [something else], so I'd like to understand why python isn't doing the same and fix the problem... A profile might help to understand what the time is used for. As almost all operations are not done in Python itself (httplib is really a very tiny wrapper above a socket), a C level profile may be necessary to understand the behaviour. Actually, the problem *was* in Python: http://bugs.python.org/issue2576 Found and fixed :-) Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: [Guppy-pe-list] An iteration idiom (Was: Re: loading files containing multiple dumps)
Sverker Nilsson wrote: I hope the new loadall method as I wrote about before will resolve this. def loadall(self,f): ''' Generates all objects from an open file f or a file named f''' if isinstance(f,basestring): f=open(f) while True: yield self.load(f) It would be great if load either returned just one result ever, or properly implemented the iterator protocol, rather than half implementing it... Should we call it loadall? It is a generator so it doesn't really load all immedietally, just lazily. Maybe call it iload? Or redefine load, but that might break existing code so would not be good. loadall works for me, iload doesn't. Minor rant, why do I have to instantiate a class 'guppy.heapy.Use._GLUECLAMP_' to do anything with heapy? Why doesn't heapy just expose load, dump, etc? Basically, the need for the h=hpy() idiom is to avoid any global variables. Eh? What's h then? (And h will reference whatever globals you were worried about, surely?) Heapy uses some rather big internal data structures, to cache such things as dict ownership. I didn't want to have all those things in global variables. What about attributes of a class instance of some sort then? the other objects you created. Also, it allows for several parallel invocations of Heapy. When is that helpful? However, I am aware of the extra initial overhead to do h=hpy(). I discussed this in my thesis. Section 4.7.8 Why not importing Use directly? page 36, http://guppy-pe.sourceforge.net/heapy-thesis.pdf I'm afraid, while I'd love to, I don't have the time to read a thesis... Try sunglasses:) (Well, I am aware of this, it was a research/experimental system and could have some refactoring :-) I would suggest creating a minimal system that allows you to do heap() and then let other people build what they need from there. Simple is *always* better... Less minor rant: this applies to most things to do with heapy... Having __repr__ return the same as __str__ and having that be a long lump of text is rather annoying. If you really must, make __str__ return the big lump of text but have __repr__ return a simple, short, item containing the class, the id, and maybe the number of contained objects... I thought it was cool to not have to use print but get the result directly at the prompt. That's fine, that's what __str__ is for. __repr__ should be short. Hmmm, I'm sure there's a good reason why an item in a set has the exact same class and iterface as a whole set? Um, perhaps no very good reason but... a subset of a set is still a set, isn't it? Yeah, but an item in a set is not a set. __getitem__ should return an item, not a subset... I really think that, by the sounds of it, what is currently implemented as __getitem__ should be a `filter` or `subset` method on IdentitySets instead... objects. Each row is still an IdentitySet, and has the same attributes. Why? It's semantically different. .load() returns a set of measurements, each measurement contains a set of something else, but I don't know what... This is also like Python strings work, there is no special character type, a character is just a string of length 1. Strings are *way* more simple in terms of what they are though... cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
uses for setup.cfg and extracting data from it
Hi All, Do people generally source control their package's setup.cfg? http://docs.python.org/distutils/configfile.html sort of implies it should be editable by the person installing the package, but I've never personally used a package where that's the case... Assuming the distutils docs are out of date and this file is really owned by the package maintainer, how do I extract information from it in setup.py (and elsewhere for that matter!) I'm looking for somewhere consistent to store the following for all my packages: - mailing lists url - issue tracker url - change log url - documentation url ...such that I can generate a sensible long_description for use on PyPI but also such that I can include the information in the Sphinx docs... cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: [Guppy-pe-list] An iteration idiom (Was: Re: loading files containing multiple dumps)
Sverker Nilsson wrote: But I don't think I would want to risk breaking someone's code just for this when we could just add a new method. I don't think anyone will be relying on StopIteration being raised. If you're worried, do the next release as a 0.10.0 release and explain the backwards incompatible change in the release announcement. Or we could have an option to hpy() to redefine load() as loadall(), but I think it is cleaner (and easier) to just define a new method... -1 to options to hpy, +1 to loadall but also -1 to lead load() as broken as it is... As the enclosing class or frame is deallocated, so is its attribute h itself. Right, but as long as the h hangs around, it hangs on to all the memory it's used to build its stats, right? This caused me problems in my most recent use of guppy... themselves, but I am talking about more severe data that can be hundreds of megabytes or more). Me too ;-) I've been profiling situations where the memory usage was over 1GB for processing a 30MB file when I started ;-) For example, the setref() method sets a reference point somewhere in h. Further calls to heap() would report only objects allocated after that call. But you could use a new hpy() instance to see all objects again. Multiple threads come to mind, where each thread would have its own hpy() object. (Thread safety may still be a problem but at least it should be improved by not sharing the hpy() structures.) Even in the absence of multiple threads, you might have an outer invocation of hpy() that is used for global analysis, with its specific options, setref()'s etc, and inner invocations that make some local analysis perhaps in a single method. Fair points :-) http://guppy-pe.sourceforge.net/heapy-thesis.pdf I'm afraid, while I'd love to, I don't have the time to read a thesis... But it is (an important) part of the documentation. That may be, but I'd wager a fair amount of beer that buy far the most common uses for heapy are: - finding out what's using the memory consumed by a python process - log how what the memory consumption is made up of while running a large python process - finding out how much memory is being used ...in that order. Usually on a very tight deadline and with unhappy users breathing down their necks. At times like that, reading a thesis doesn't really figure into it ;-) I'm afraid, while I'd love to, I don't have the time to duplicate the thesis here...;-) I don't think that would help. Succinct help and easy to use functions to get those 3 cases above solved is all that's needed ;-) Do you mean we should actually _remove_ features to create a new standalone system? Absolutely, why provide more than is used or needed? You are free to wrap functions as you find suitable; a minimal wrapper module could be just like this: # Module heapyheap from guppy import hpy h=hpy() heap=heap() I don't follow this.. did you mean heap = h.heap()? If so, isn't that using all the gubbinz in Use, etc, anyway? Less minor rant: this applies to most things to do with heapy... Having __repr__ return the same as __str__ and having that be a long lump of text is rather annoying. If you really must, make __str__ return the big lump of text but have __repr__ return a simple, short, item containing the class, the id, and maybe the number of contained objects... I thought it was cool to not have to use print but get the result directly at the prompt. That's fine, that's what __str__ is for. __repr__ should be short. No, it's the other way around: __repr__ is used when evaluating directly at the prompt. The docs give the idea: http://docs.python.org/reference/datamodel.html?highlight=__repr__#object.__repr__ I believe you big strings would be classed as informal and so would be computed by __str__. Yeah, but an item in a set is not a set. __getitem__ should return an item, not a subset... Usually I think it is called an 'element' of a set rather than an 'item'. Python builtin sets can't even do indexing at all. ...'cos it doesn't make sense ;-) Likewise, Heapy IdentitySet objects don't support indexing to get at the elements directly. ...then they shouldn't have a __getitem__ method! The index (__getitem__) method was available so I used it to take the subset of the i'ths row in the partition defined by its equivalence order. That should have another name... I don't know what a partition or equivalence order are in the contexts you're using them, but I do know that hijacking __getitem__ for this is wrong. The subset indexing, being the more well-defined operation, and also IMHO more generally useful, thus got the honor to have the [] syntax. Except it misleads anyone who's programmed in Python for a significant period of time and causes problems when combined with the bug in .load :-( It would just be another syntax. I don't see the conceptual problem since e.g. indexing works just fine like this with
Re: Some issue with easy_install and PIL/Imaging
Klein Stéphane wrote: Resume : 1. first question : why PIL package in pypi don't work ? Because Fred Lundh have his package distributions unfortunate names that setuptools doesn't like... 2. second question : when I add PIL dependence in my setup.py and I do python setup.py develop, I've this error : error: Could not find required distribution Imaging. Why ? See above. Now, I add --find-links parameter to easy_install : If you google harder, you'll find there are packagings of PIL that do work with setuptools... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: urlopen error (11001, 'getaddrinfo failed')
open...@hushmail.com wrote:
fs = cgi.FieldStorage()
url = fs.getvalue('url', http://www.openlayers.org;)
try:
insert a print url here...
y = urllib2.urlopen(url)
print y.read()
This script produces the urlopen error (11001, 'getaddrinfo
failed').
This is a name lookup failing, whatever you're ending up with in url is
either incorrect or the dns config of the server it's pointing at is
incorrect...
Chris
--
Simplistix - Content Management, Batch Processing Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list
Re: [Guppy-pe-list] An iteration idiom (Was: Re: loading files containing multiple dumps)
Sverker Nilsson wrote: If you just use heap(), and only want total memory not relative to a reference point, you can just use hpy() directly. So rather than: CASE 1: h=hpy() h.heap().dump(...) #other code, the data internal to h is still around h.heap().dump(...) you'd do: CASE 2: hpy().heap().dump(...) #other code. No data from Heapy is hanging around hpy().heap().dump(...) The difference is that in case 1, the second call to heap() could reuse the internal data in h, But that internal data would have to hang around, right? (which might, in itself, cause memory problems?) whereas in case 2, it would have to be recreated which would take longer time. (The data would be such things as the dictionary owner map.) How long is longer? Do you have any metrics that would help make good decisions about when to keep a hpy() instance around and when it's best to save memory? Do you mean we should actually _remove_ features to create a new standalone system? Absolutely, why provide more than is used or needed? How should we understand this? Should we have to support 2 or more systems depending on what functionality you happen to need? Or do you mean most functionality is actually _never_ used by _anybody_ (and will not be in the future)? That would be quite gross wouldn't it. I'm saying have one project and dump all the excess stuff that no-one but you uses ;-) Or, maybe easier, have a core, separate, package that just has the essentials in a simply, clean fashion and then another package that builds on this to add all the other stuff... It also gives as an alternative, If this is not possible, a string of the form ...some useful description... should be returned The __repr__ I use don't have the enclosing , granted, maybe I missed this or it wasn't in the docs in 2005 or I didn't think it was important (still don't) but was that really what the complain was about? No, it was about the fact that when I do repr(something_from_heapy) I get a shedload of text. I thought it was more useful to actually get information of what was contained in the object directly at the prompt, than try to show how to recreate it which wasn't possible anyway. Agreed, but I think the stuff you currently have in __repr__ would be better placed in its own method: heap() IdentitySet object at 0x containing 10 items _.show() ... all the current __repr__ output That should have another name... I don't know what a partition or equivalence order are in the contexts you're using them, but I do know that hijacking __getitem__ for this is wrong. Opinions may differ, I'd say one can in principle never 'know' if such a thing is 'right' or 'wrong', but that gets us into philosophical territory. Anyway... I would bet that if you asked 100 experienced python programmers, most of them would tell you that what you're doing with __getitem__ is wrong, some might even say evil ;-) To get a tutorial provided by someone who did not seem to share your conviction about indexing, but seemed to regard the way Heapy does it natural (although has other valid complaints, though it is somewhat outdated i.e. wrt 64 bit) see: http://www.pkgcore.org/trac/pkgcore/doc/dev-notes/heapy.rst This link has become broken recently, but I don't remember reading the author's comments as liking the indexing stuff... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for a pure Python chart drawing module
John Nagle wrote: That's a wrapper for Antigrain (http://www.antigrain.com/;), which is a C++ library. I'm trying hard to avoid dependencies on binary libraries with limited support. Builds exist only for Python 2.4 and 2.5. Huh? Matplotlib is a pretty phenomenal charting library, I use it routinely on both windows and linux, I've never had any compilation problems on Linux and never even needed to compile it on Windows. Writing if off as just a wrapper for antigrain is pretty insulting... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for a pure Python chart drawing module
Grant Edwards wrote: On 2009-09-16, Alan G Isaac alan.is...@gmail.com wrote: Tkinter is part of the Python standard library: That doesn't mean you can depend on it being available. It doesn't get installed by default on some Linux distros. That's 'cos some linux distros feel the need to destroy python's packaging for their own silly reasons... Take them out and shoot them. Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: recommendation for webapp testing?
Simon Brunning wrote: Mechanize is a superb library for its intended purpose - I use it all the time. It's lack of support for pages with JavaScript functionality, though, means it's not very useful at a testing tool for modern web sites. There's also zope.testbrowser, which is a handy wrapper around Mechanize. I wonder if they've done anything with JS there? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I begin debugging a python memory leak?
Matthew Wilson wrote: I have a web app based on TurboGears 1.0. In the last few days, as traffic and usage has picked up, I noticed that the app went from using 4% of my total memory all the way up to 50%. I suspect I'm loading data from the database and somehow preventing garbage collection. Are there any tools that might help me out with this? http://pypi.python.org/pypi/guppy ...which will give you: http://guppy-pe.sourceforge.net/heapy_tutorial.html Good for finding out what's making up the memory usage. cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
TestFixtures 1.6.2 released!
Hi All, I'm pleased to announce a new release of TestFixtures. This package is a collection of helpers and mock objects that are useful when writing unit tests or doc tests. This release fixes problems when using Comparison objects with instances of Django models, so tests like the following will now work as expected: from testfixtures import Comparison as C,compare class TestUser(TestCase): def test_create(self): u = User(name='Test') u.save() t = User.objects.get(name='Test') compare([C(User,name='Test',strict=False)], list(User.objects.all())) To find out more, please read here: http://pypi.python.org/pypi/testfixtures cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
problems trying to build python 2.6 as a shared library
Hi All, I'm trying to build Python 2.6 as a shared library, so I did: ./configure --enable-shared make make altinstall No obvious signs of failure, but when I try and use the resulting python, I get: python2.6: error while loading shared libraries: libpython2.6.so.1.0: cannot open shared object file: No such file or directory Why might that be? cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
problems compiling pyscopg2 on RHEL4
Hi All, I installed the following rpms: postgresql-libs-7.4.19-1.el4_6.1 postgresql-devel-7.4.19-1.el4_6.1 postgresql-7.4.19-1.el4_6.1 And then installed psycopg2 as follows: wget http://initd.org/pub/software/psycopg/psycopg2-2.0.12.tar.gz tar xzf psycopg2-2.0.12.tar.gz cd psycopg2-2.0.12 python2.6 setup.py install However, it appears something went wrong: # python2.6 Python 2.6.2 (r262:71600, Sep 25 2009, 12:03:16) [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] on linux2 Type help, copyright, credits or license for more information. import psycopg2 Traceback (most recent call last): File stdin, line 1, in module File /usr/local/lib/python2.6/site-packages/psycopg2/__init__.py, line 60, in module from _psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID ImportError: /usr/local/lib/python2.6/site-packages/psycopg2/_psycopg.so: undefined symbol: PQserverVersion Has anyone seen this and/or know how to fix it? cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: logging.handlers.SMTPHandler question
akonsu wrote: hello, SMTPHAndler seems to email every single record separately. is there a way to collect all log output and then send it in a single email message? or do i have to do it manually? You want the SummarisingHandler from this package: http://www.simplistix.co.uk/software/python/mailinglogger cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Repeated output when logging exceptions
Vinay Sajip wrote: I'm not sure why you need all the code you've posted. The logging package allows you to add tracebacks to your logs by using the exception() method, which logs an ERROR with a traceback and is specifically intended for use from within exception handlers. You can also use the exc_info=True parameter with any logging method... Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
Re: Repeated output when logging exceptions
John Gordon wrote: If I didn't do all that in a class, where would I do it? I find the configureLoggers method of ZConfig most convenient for this: http://pypi.python.org/pypi/ZConfig cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- http://mail.python.org/mailman/listinfo/python-list
