[ANN] Leipzig Python User Group - Meeting, January 12, 2010, 08:00pm
=== Leipzig Python User Group === We will meet on Tuesday, January 12 at 8:00 pm at the training center of Python Academy in Leipzig, Germany ( http://www.python-academy.com/center/find.html ). Markus Zapke-Gründemann will introduce the modules xml.sax[1] and xml.sax.handler[2]. They are useful for fast processing of large amounts of XML data. Food and soft drinks are provided. Please send a short confirmation mail to i...@python-academy.de, so we can prepare appropriately. Everybody who uses Python, plans to do so or is interested in learning more about the language is encouraged to participate. While the meeting language will be mainly German, we will provide English translation if needed. Current information about the meetings are at http://www.python-academy.com/user-group . Mike == Leipzig Python User Group === Wir treffen uns am Dienstag, 12.01.2010 um 20:00 Uhr im Schulungszentrum der Python Academy in Leipzig ( http://www.python-academy.de/Schulungszentrum/anfahrt.html ). Markus Zapke-Gründemann wird die Module xml.sax[1] und xml.sax.handler[2] vorstellen, mit denen sich schnell große Mengen an XML-Daten verarbeiten lassen. Für das leibliche Wohl wird gesorgt. Eine Anmeldung unter i...@python-academy.de wäre nett, damit wir genug Essen besorgen können. Willkommen ist jeder, der Interesse an Python hat, die Sprache bereits nutzt oder nutzen möchte. Aktuelle Informationen zu den Treffen sind unter http://www.python-academy.de/User-Group zu finden. Viele Grüße Mike [1] http://docs.python.org/library/xml.sax.html [2] http://docs.python.org/library/xml.sax.handler.html -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
SQLObject 0.11.3
Hello! I'm pleased to announce version 0.11.3, a minor bugfix release of 0.11 branch of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.11.3 News and changes: http://sqlobject.org/News.html What's New == News since 0.11.2 - * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
SQLObject 0.12.1
Hello! I'm pleased to announce version 0.12.1, a bugfix release of branch 0.12 of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.12.1 News and changes: http://sqlobject.org/News.html What's New == News since 0.12.0 - * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed three bugs in PostgresConnection. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Caching objects in a C extension
I'm working with a C extension that needs to rapidly create and delete
objects. I came up with an approach to cache objects that are being
deleted and resurrect them instead of creating new objects. It appears
to work well but I'm afraid I may be missing something (besides
heeding the warning in the documentation that _Py_NewReference is for
internal interpreter use only).
Below is a simplified version of the approach I'm using:
MyType_dealloc(MyTypeObject *self)
{
if(I_want_to_save_MyType(self)) {
// Save the object pointer in a cache
save_it(self);
} else {
PyObject_Del(self);
}
}
MyType_new(void)
{
MyTypeObject *self;
if(there_is_an_object_in_the_cache) {
self = get_object_from_cache;
_Py_NewReference((PyObject*)self);
} else {
if(!(self = PyObjectNew(MyTypeObject, MyType))
return NULL;
initialize_the_new_object(self);
}
return self;
}
The objects referenced in the cache have a reference count of 0 and I
don't increment the reference count until I need to resurrect the
object. Could these objects be clobbered by the garbage collector?
Would it be safer to create the new reference before stuffing the
object into the cache (even though it will look like there is a memory
leak when running under a debug build)?
Thanks in advance for any comments,
casevh
--
http://mail.python.org/mailman/listinfo/python-list
Re: Ask how to use HTMLParser
On 8 янв, 11:44, Water Lin water...@ymail.invalid wrote: h0uk vardan.pogos...@gmail.com writes: On 8 янв, 08:44, Water Lin water...@ymail.invalid wrote: I am a new guy to use Python, but I want to parse a html page now. I tried to use HTMLParse. Here is my sample code: -- from HTMLParser import HTMLParser from urllib2 import urlopen class MyParser(HTMLParser): title = is_title = def __init__(self, url): HTMLParser.__init__(self) req = urlopen(url) self.feed(req.read()) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print Found link = %s % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print here self.title = data print self.title self.is_title = 0 --- For the tag --- div class=articleTitleopen article title/div --- I use my code to parse it. I can locate the div tag but I don't know how to get the text for the tag which is open article title in my example. How can I get the html content? What's wrong in my handle_data function? Thanks Water Lin -- Water Lin's notes and pencils:http://en.waterlin.org Email: water...@ymail.com I want to say your code works well But in handle_data I can't print self.title. I don't why I can't set the self.title in handle_data. Thanks Water Lin -- Water Lin's notes and pencils:http://en.waterlin.org Email: water...@ymail.com I have tested your code as : #!/usr/bin/env python # -*- conding: utf-8 -*- from HTMLParser import HTMLParser class MyParser(HTMLParser): title = is_title = def __init__(self, data): HTMLParser.__init__(self) self.feed(data) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print Found link = %s % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print here self.title = data print self.title self.is_title = 0 if __name__ == __main__: m = MyParser( div class=secttlbarwrap table cellpadding=0 cellspacing=0 width=100%trtd div style=background: url(/groups/roundedcorners? c=99bc=whitew=4h=4a=af) 0px 0px; width: 4px; height: 4px td bgcolor=#99 width=100% height=4img alt= width=1 height=1td div style=background: url(/groups/roundedcorners? c=99bc=whitew=4h=4a=af) -4px 0px; width: 4px; height: 4px /div/table/div div class=articleTitleopen article title/div div class=secttlbar div class=lf secttl span id=thread_subject_site Ask how to use HTMLParser /span /div div class=rf secmsg frtxt padt2 a class=uitl id=showoptions_lnk2 href=# onclick=TH_ToggleOptionsPane(); return false;Parametrs/a /div div class=hght0 clear style=font-size:0;/div /div) All stuff printed and handled fine. Also, the 'print self.title' statement works fine. Try run my code. Vardan. -- http://mail.python.org/mailman/listinfo/python-list
Re: C Structure rebuild with ctypes
Hi Mark, Are you passing in these values, or are they being returned? To me the depth of the pointer references implies numVars, varNames, and varTypes are out parameters. I'll assume that for now. If they are in/out parameters let me know. If these parameters were in parameters. What would I have to do different? Best regards Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
On Wed, Jan 6, 2010 at 2:19 PM, Victor Subervi victorsube...@gmail.comwrote:
On Wed, Jan 6, 2010 at 3:09 PM, Carsten Haese carsten.ha...@gmail.comwrote:
Victor Subervi wrote:
I have an automatically generated HTML form from which I need to extract
data to the script which this form calls (to which the information is
sent).
Ideally, the script that receives the submitted fields should know how
the form was generated, so it knows what fields to expect. Failing that,
you should realize that getfirst() is not the only way to access the
contents of a cgi.FieldStorage object.
If you need to know the values of all fields whose names obey a certain
pattern, I recommend you start with a list of all field names in the
FieldStorage object and pick out the ones whose names obey that pattern.
To get the list of field names, you should consider using the .keys()
method of the FieldStorage object.
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
return params
def display():
top()
dict = cgiFieldStorageToDict(cgi.FieldStorage())
print dict
Error:
/var/www/html/angrynates.com/christians/cart/display.py
163 print 'All products together usually cost: $%fbr /br /' %
str(fpformat.fix(round(int(allPrices * 100))/100, 2))
164 cursor.close()
165 bottom()
166
167 display()
display = function display
/var/www/html/angrynates.com/christians/cart/display.py in display()
120 def display():
121 top()
122 dict = cgiFieldStorageToDict(cgi.FieldStorage())
123 print dict
124 if store == 'prescriptions':
builtin dict = type 'dict', global cgiFieldStorageToDict = function
cgiFieldStorageToDict, global cgi = module 'cgi' from
'/usr/lib64/python2.4/cgi.pyc', cgi.FieldStorage = class cgi.FieldStorage
/var/www/html/angrynates.com/christians/cart/display.py in
cgiFieldStorageToDict(fieldStorage=FieldStorage(None, None,
[MiniFieldStorage('stor...products'), MiniFieldStorage('cat', 'prodCat1')]))
115 params = {}
116 for key in fieldStorage.keys():
117 params[key] = cgi.FieldStorage[key].value
118 return params
119
params = {}, key = 'store', global cgi = module 'cgi' from
'/usr/lib64/python2.4/cgi.pyc', cgi.FieldStorage = class
cgi.FieldStorage, ].value undefined
TypeError: unsubscriptable object
args = ('unsubscriptable object',)
What do?
TIA,
beno
#! /usr/bin/python
import cgitb; cgitb.enable()
import cgi
import MySQLdb
import sys,os
sys.path.append(os.getcwd())
from login import login
from template import top, bottom
from sets import Set
import fpformat
form = cgi.FieldStorage()
store = form.getfirst('store')
cat = form.getfirst('cat', '')
user, passwd, db, host = login()
db = MySQLdb.connect(host, user, passwd, db)
cursor = db.cursor()
def displayProducts(patientID=''):
try: # These are stores with categories where ordering by price is
important
sql = 'select ID from %s where Category=%s order by Price desc;' %
(store, cat)
cursor.execute(sql)
except: # Stores, like prescriptions, where ordering by price is not
important
sql = 'select ID from %s;' % (store)
cursor.execute(sql)
ids = [itm[0] for itm in cursor]
cursor.execute('describe %s;' % store)
colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for itm in
cursor]
i = 0
if len(ids) 0:
for id in ids:
# print 'tr\n'
print 'form method=post action=displayOneProduct.py'
print input type='hidden' name='store' value='%s' / % store
print input type='hidden' name='id' value='%s' / % id
j = 0
for col in colFields:
sql = 'select %s from %s where ID=%s;' % (col, store, str(id))
cursor.execute(sql)
colValue = cursor.fetchone()
if col == 'SKU':
print input type='hidden' name='sku' value='%s' / %
colValue[0]
print 'b%s: /b%sbr /\n' % (col, colValue[0])
elif col == 'Category':
print input type='hidden' name='cat' value='%s' / %
colValue[0]
print 'b%s: /b%sbr /\n' % (col, colValue[0])
elif col == 'OutOfStock':
if colValue[0] == '1': # This product is out of stock
outOfStockFlag = 'yes'
else:
outOfStockFlag = 'no'
elif col[:3] != 'pic':
notSet = 1
if isinstance(colValue[0], (str, int, float, long, complex,
unicode, list, buffer, xrange, tuple)):
pass
else:
try:
html = b%s/b: select name='%s' % (col, col)
notSet = 0
for itm in colValue[0]:
try:
color, number = itm.split('$')
html += option name='%s'%s/option % (itm, color)
except:
html += option name='%s'%s/option % (itm, itm)
html += /selectbr /
print html
except:
pass
if notSet == 1:
multiprocessing and remote objects
Hi all
I am experimenting with the multiprocessing module, to get a feel of what is
possible and what is not. I have got a situation that does not behave as
expected. I can work around it, but I would like to find out the underlying
reason, to get a more solid understanding.
I am trying to create 'remote objects' that can be shared between processes.
I can create and return an instance of a class, with no problem. Now I want
to add a method to the class which, when called remotely, creates and
returns an instance of another class. I cannot get this to work.
On the server side I have -
tables = {} # a cache of table instances, keyed on name
class Table(object):
def __init__(self, name):
self.name = name
def get_table(self, tablename):
# if table with this name does not exist,
# create it and store it in cache
if tablename not in tables:
tables[tablename] = Table(tablename)
# retrieve table instance from cache and return it
return tables[tablename]
class MyManager(BaseManager): pass
MyManager.register('get_table', get_table,
method_to_typeid={'get_table': 'Table')
MyManager.register('Table', Table)
if __name__ == '__main__':
manager = MyManager(address=('127.0.0.1', 5))
server = manager.get_server()
server.serve_forever()
On the client side I have -
class MyManager(BaseManager): pass
MyManager.register('get_table')
MyManager.register('Table')
if __name__ == '__main__':
manager = MyManager(address=('127.0.0.1', 5))
manager.connect()
cust = manager.get_table('Customers')
So far so good.
Then I extend the server program as follows -
class Table(object):
def __init__(self, name):
self.name = name
+ self.columns = {}
+ def add_column(self, colname):
+ if colname not in self.columns:
+ self.columns[colname] = Column(colname)
+ return self.columns[colname]
+ class Column(object):
+ def __init__(self, colname):
+ self.colname = colname
MyManager.register('get_table', get_table,
method_to_typeid={'get_table': 'Table')
- MyManager.register('Table', Table)
+ MyManager.register('Table', Table,
+ method_to_typeid={'add_column': 'Column')
+ MyManager.register('Column', Column)
and I extend the client program as follows -
MyManager.register('get_table')
MyManager.register('Table')
+ MyManager.register('Column')
cust = manager.get_table('Customers')
+ acno = cust.add_column('Acno')
The server appears to create and return the column ok, but on the client
side I get the following traceback -
Traceback (most recent call last):
File F:\junk\multiprocess\mp13b.py, line 29, in module
acno = cust.add_column('Acno')
File string, line 2, in add_column
File C:\Python26\lib\multiprocessing\managers.py, line 726, in
_callmethod
kind, result = conn.recv()
AttributeError: 'module' object has no attribute 'Column'
Does anyone know if it is possible to do what I am attempting, and what the
correct approach is?
BTW, version is Python 2.6.2.
Thanks
Frank Millman
--
http://mail.python.org/mailman/listinfo/python-list
Standardized interpreter speed evaluation tool
Is there any standardized interpreter speed evaluation tool? Say I made few changes in interpreter code and want to know if those changes made python any better, which test should I use? -- http://mail.python.org/mailman/listinfo/python-list
Re: Ask how to use HTMLParser
Water Lin wrote: h0uk vardan.pogos...@gmail.com writes: On 8 янв, 08:44, Water Lin water...@ymail.invalid wrote: I am a new guy to use Python, but I want to parse a html page now. I tried to use HTMLParse. Here is my sample code: -- from HTMLParser import HTMLParser from urllib2 import urlopen class MyParser(HTMLParser): title = is_title = def __init__(self, url): HTMLParser.__init__(self) req = urlopen(url) self.feed(req.read()) def handle_starttag(self, tag, attrs): if tag == 'div' and attrs[0][1] == 'articleTitle': print Found link = %s % attrs[0][1] self.is_title = 1 def handle_data(self, data): if self.is_title: print here self.title = data print self.title self.is_title = 0 --- For the tag --- div class=articleTitleopen article title/div --- I use my code to parse it. I can locate the div tag but I don't know how to get the text for the tag which is open article title in my example. How can I get the html content? What's wrong in my handle_data function? Thanks Water Lin -- Water Lin's notes and pencils:http://en.waterlin.org Email: water...@ymail.com I want to say your code works well But in handle_data I can't print self.title. I don't why I can't set the self.title in handle_data. Thanks Water Lin I don't know HTMLParser, but I see a possible confusion point in your class definition. You have both class-attributes and instance-attributes of the same names (title and is_title). So if you have more than one instance of MyParser, then they won't see each other's changes. Normally, I'd move the initialization of such attributes into the __init__() method, so the behavior is clear. When an instance-attribute has the same name as a class-attribute, the instance-attribute takes precedence, and hides the class-attribute, for further processing in that same instance. So effectively, the class-attribute acts as a default value. -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I access what's in this module?
On 2010-01-08 05:01, John Machin wrote: Error message should appear after line starting with File. Above excerpt taken from google groups; identical to what shows in http://news.gmane.org/gmane.comp.python.general ... what are you looking at? With Windows XP and Python 2.5.4 I get: Traceback (most recent call last): File stdin, line 1, inmodule AttributeError: 'builtin_function_or_method' object has no attribute 'dump' I'm sorry, I managed to leave out that last line by mistake! My bad. I didn't spot that in my first reply to you because I was under the impression that you hadn't seen the tiniest part of traceback. As you neatly pointed out earlier, it's easy to become confused when communicating. :) It turns out I no longer want to access anything in there but I thank you for your information nontheless. You're welcome -- the advice on _methods is portable :-) I will look more closely at what other advice you write, I must confess I didn't actually do that because, as I mentioned, I no longer had any interest in accessing the module and I was busy (and still am) with another problem. Thanks John! - Fencer -- http://mail.python.org/mailman/listinfo/python-list
Re: Standardized interpreter speed evaluation tool
alexru schrieb: Is there any standardized interpreter speed evaluation tool? Say I made few changes in interpreter code and want to know if those changes made python any better, which test should I use? Pybench I guess. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Caching objects in a C extension
casevh schrieb:
I'm working with a C extension that needs to rapidly create and delete
objects. I came up with an approach to cache objects that are being
deleted and resurrect them instead of creating new objects. It appears
to work well but I'm afraid I may be missing something (besides
heeding the warning in the documentation that _Py_NewReference is for
internal interpreter use only).
Below is a simplified version of the approach I'm using:
MyType_dealloc(MyTypeObject *self)
{
if(I_want_to_save_MyType(self)) {
// Save the object pointer in a cache
save_it(self);
} else {
PyObject_Del(self);
}
}
MyType_new(void)
{
MyTypeObject *self;
if(there_is_an_object_in_the_cache) {
self = get_object_from_cache;
_Py_NewReference((PyObject*)self);
} else {
if(!(self = PyObjectNew(MyTypeObject, MyType))
return NULL;
initialize_the_new_object(self);
}
return self;
}
The objects referenced in the cache have a reference count of 0 and I
don't increment the reference count until I need to resurrect the
object. Could these objects be clobbered by the garbage collector?
Would it be safer to create the new reference before stuffing the
object into the cache (even though it will look like there is a memory
leak when running under a debug build)?
Deep out of my guts I'd say keeping a reference, and using you own
LRU-scheme would be the safest without residing to use dark magic.
Diez
--
http://mail.python.org/mailman/listinfo/python-list
Re: Standardized interpreter speed evaluation tool
alexru wrote: Is there any standardized interpreter speed evaluation tool? Say I made few changes in interpreter code and want to know if those changes made python any better, which test should I use? Not trying to be a smart-aleck, but the test you use should reflect your definition of the phrase any better. For example, suppose you decided that you could speed things up by pre-calculating a few dozen megabytes of data, and including that in the python.dll. This would change the memory footprint of Python, and possibly the interpreter's startup/load time, not just the runtime of whatever loop you are testing. But if I assume you know all that and just want to do timings. There are at least two stdlib functions that can help: import time time.time() and time.clock() will give you a wall-clock floating point number, and you can subtract two of these within the same program to see how long something takes. This approach ignores interpreter startup, and does nothing to compensate for other processes running on the system. But it can be very useful, and easy to run. The resolution on each function varies by OS, so you may have to experiment to see which one gives the most precision. import time start = time.time() dotest() print Elapsed time, time.time() - start timeit module can be used within code for timing, or it may be used to load and run a test from the command line. In the latter version, it includes the startup time for the interpreter, which might be important. It also can execute the desired code repeatedly, so you can get some form of averaging, or amortizing. *** python -m timeit Note that due to system caching, doing multiple runs consecutively may give different results than ones that are separated by other programs running. And of course when you recompile and link., the system buffers will contain an unusual set of data. So there are ways (which I don't recall) of flushing the system buffers to let programs start on equal footing. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: PyQt QThreadPool error
On Thu, 7 Jan 2010 15:07:10 -0800 (PST), h0uk vardan.pogos...@gmail.com wrote: ... Phil you right about app.exec_(). But situation is sligthly different. I want to have more than one Job. I add these Jobs into QThreadPool trough cycle. And I also want these Jobs to run sequentially. The following code illustrate what I mean: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys import os import time from PyQt4 import QtCore, QtGui class Job(QtCore.QRunnable): def __init__(self, name): QtCore.QRunnable.__init__(self) self._name = name def run(self): time.sleep(3) print self._name if __name__ == __main__: app = QtGui.QApplication(sys.argv) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) for i in range(5): j = Job(Job- + str(i)) j.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(j, i) app.exec_() After 5 cycle I get the same error: An unhandled win32 exception occured in python.exe. How I can do it?? To run my Jobs sequentially??? It's a PyQt bug. The workaround is not to use setAutoDelete() and instead keep an explicit reference to your Job instances - for example in a list of jobs. Phil -- http://mail.python.org/mailman/listinfo/python-list
Transforming a List of elements into a List of lists of elements
Hello all, I'd like to ask how you'd implement a function *f* that transforms a list of elements into a list of lists of elements by grouping contiguous elements together. Examples: a=[1,2,3,4,5] print( f(a, 2) ) # - [ [1, 2], [3, 4], [5] ] print( f(a, 3) ) # - [ [1, 2, 3], [4, 5] ] print( f(a, 1) ) # - [ [1], [2], [3], [4], [5] ] I have done a small function that does this using a while loop but what I'm looking for is a more *pythonic* way to do this. Maybe some function in some standard Module does this and I don't know? Ps: I don't post the function here because I don't have it with me. Thanks a lot in advance! Tiago Almeida tiago.b.alme...@gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Transforming a List of elements into a List of lists of elements
08-01-2010 tiago almeida tiago.b.alme...@gmail.com wrote: Hello all, I'd like to ask how you'd implement a function *f* that transforms a list of elements into a list of lists of elements by grouping contiguous elements together. Examples: a=[1,2,3,4,5] print( f(a, 2) ) # - [ [1, 2], [3, 4], [5] ] print( f(a, 3) ) # - [ [1, 2, 3], [4, 5] ] print( f(a, 1) ) # - [ [1], [2], [3], [4], [5] ] I have done a small function that does this using a while loop but what I'm looking for is a more *pythonic* way to do this. Maybe some function in some standard Module does this and I don't know? [In Python 2.x] s = [1,2,3,4,5] zip(*[iter(s)]*3) # zip truncates result [(1, 2, 3)] zip(*[iter(s)]*2) [(1, 2), (3, 4)] zip(*[iter(s)]*3) # or simpler: zip(s) :-) [(1,), (2,), (3,), (4,), (5,)] map(None, *[iter(s)]*2) # map fills result with None [(1, 2), (3, 4), (5, None)] map(None, *[iter(s)]*3) [(1, 2, 3), (4, 5, None)] Instead of map, you can use izip_longest from itertools module: list(itertools.izip_longest(*[iter(s)]*3)) [(1, 2, 3), (4, 5, None)] list(itertools.izip_longest(*[iter(s)]*2)) [(1, 2), (3, 4), (5, None)] See: http://docs.python.org/library/functions.html#zip http://docs.python.org/library/functions.html#map http://docs.python.org/library/itertools.html#itertools.izip_longest Cheers, *j -- Jan Kaliszewski (zuo) z...@chopin.edu.pl -- http://mail.python.org/mailman/listinfo/python-list
Re: Transforming a List of elements into a List of lists of elements
tiago almeida wrote: Hello all, I'd like to ask how you'd implement a function /f/ that transforms a list of elements into a list of lists of elements by grouping contiguous elements together. Examples: a=[1,2,3,4,5] print( f(a, 2) ) # - [ [1, 2], [3, 4], [5] ] print( f(a, 3) ) # - [ [1, 2, 3], [4, 5] ] print( f(a, 1) ) # - [ [1], [2], [3], [4], [5] ] I have done a small function that does this using a while loop but what I'm looking for is a more /pythonic/ way to do this. Maybe some function in some standard Module does this and I don't know? Ps: I don't post the function here because I don't have it with me. Thanks a lot in advance! Tiago Almeida tiago.b.alme...@gmail.com mailto:tiago.b.alme...@gmail.com using list comprehention (http://docs.python.org/tutorial/datastructures.html) # note: poor naming def func(l, slice): r = range(0, len(l), slice) result = [l[sl:sl+slice] for sl in r] print result a=[1,2,3,4,5] func(a,1) ; func(a,2) ; func(a,3) [[1], [2], [3], [4], [5]] [[1, 2], [3, 4], [5]] [[1, 2, 3], [4, 5]] I would add that I personally find list comprehension hard to read, so I'm not sure it is that pythonic. Your old school while loop could be less concise/elegant, but perfectly readable. Jean-Michel -- http://mail.python.org/mailman/listinfo/python-list
Re: Transforming a List of elements into a List of lists of elements
PS. Sorry, I wrote: zip(*[iter(s)]*3) # or simpler: zip(s) :-) But should be zip(*[iter(s)]*1) # or simpler: zip(s) :-) *j -- http://mail.python.org/mailman/listinfo/python-list
SQLObject 0.11.3
Hello! I'm pleased to announce version 0.11.3, a minor bugfix release of 0.11 branch of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.11.3 News and changes: http://sqlobject.org/News.html What's New == News since 0.11.2 - * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-list
SQLObject 0.12.1
Hello! I'm pleased to announce version 0.12.1, a bugfix release of branch 0.12 of SQLObject. What is SQLObject = SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject == Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/0.12.1 News and changes: http://sqlobject.org/News.html What's New == News since 0.12.0 - * The cache culling algorithm was enhanced to eliminate memory leaks by removing references to dead objects; tested on a website that runs around 4 million requests a day. * Fixed a bug in col.py and dbconnection.py - if dbEncoding is None suppose it's 'ascii'. * Fixed three bugs in PostgresConnection. * Fixed a bug in FirebirdConnection. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytmanhttp://phd.pp.ru/p...@phd.pp.ru Programmers don't die, they just GOSUB without RETURN. -- http://mail.python.org/mailman/listinfo/python-list
dtd validation on python 3.1 on Windows.
What is the best way to validate xmls against my dtd in python? I dont see minodom doing this. Sax has dtd validation, but does not complain on dtd violations. (it does access the dtd, if it does not find dtd it complains me.) I am using python 3.1 on Win32 machine. I was planning to try lxml2.2.4 but it shows DLL importerror. I posted the problem on this forum a few days back, but there is no reply on that. Any help?! Regards Ashish -- http://mail.python.org/mailman/listinfo/python-list
Re: Need help to pass self.count to other classes.
Steven D'Aprano wrote: [... points out my misapprehension ...] kbi = kbInterface() sys.ps1 = kbi.prompt1 bound method kbInterface.prompt1 of __main__.kbInterface object at 0xb7cbd52cprint Hello Hello bound method kbInterface.prompt1 of __main__.kbInterface object at 0xb7cbd52c Right, this is expert mode ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS:http://holdenweb.eventbrite.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: How to execute a script from another script and other script does not do busy wait.
On Thu, 2010-01-07, danmcle...@yahoo.com wrote: On Jan 7, 9:18 am, Jorgen Grahn grahn+n...@snipabacken.se wrote: On Thu, 2010-01-07, Rajat wrote: I want to run a python script( aka script2) from another python script (aka script1). While script1 executes script2 it waits for script2 to complete and in doing so it also does some other useful work.(does not do a busy wait). My intention is to update a third party through script1 that script2 is going to take longer. I do not understand that sentence. What are you trying to do, more exactly? The best solution can be threads, os.popen, os.system or something different -- depending on the details of what you want to do. ... I personally use subprocess. Once you launch via subprocess you can wait or not. p = subprocess.Popen(...) p.wait() #or not. See subprocess docs. Yes, that was included in or something different above. I have never used it myself, since I've needed to be compatible with Python 2.4. Still, we need to know what he tries to do. /Jorgen -- // Jorgen Grahn grahn@ Oo o. . . \X/ snipabacken.se O o . -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote: Code snippet: [...] Error: [...] What do? After eliminating the pieces of your post that have been copied or quoted from elsewhere, I am left with a total of five words of your own to ask this question, which seems to be approximately the same amount of effort you have put into trying to figure out what's causing the error. If you put in a little bit more effort than that, you might figure out yourself what's causing the error, which should then logically lead you to how to fix the error. Good luck. -Carsten -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
On Fri, Jan 8, 2010 at 10:11 AM, Carsten Haese carsten.ha...@gmail.comwrote:
Victor Subervi wrote:
Code snippet:
[...]
Error:
[...]
What do?
After eliminating the pieces of your post that have been copied or
quoted from elsewhere, I am left with a total of five words of your own
to ask this question, which seems to be approximately the same amount of
effort you have put into trying to figure out what's causing the error.
First I get scolded for not including enough information. Now I get scolded
for including too much. Seems I can't win. I *am* putting effort into
understanding this. I'm sorry I'm not as sharp as you are in these matters.
params = {}, key = 'store', global cgi = module 'cgi' from
'/usr/lib64/python2.4/cgi.pyc'
, cgi.FieldStorage = class cgi.FieldStorage, ].value undefined
TypeError: unsubscriptable object
args = ('unsubscriptable object',)
Why is the value undefined? What is an unsubscriptable object?
TIA,
beno
--
http://mail.python.org/mailman/listinfo/python-list
Re: Do I have to use threads?
On Wed, 2010-01-06, Gary Herron wrote: aditya shukla wrote: Hello people, I have 5 directories corresponding 5 different urls .I want to download images from those urls and place them in the respective directories.I have to extract the contents and download them simultaneously.I can extract the contents and do then one by one. My questions is for doing it simultaneously do I have to use threads? Please point me in the right direction. Thanks Aditya You've been given some bad advice here. First -- threads are lighter-weight than processes, so threads are probably *more* efficient. However, with only five thread/processes, the difference is probably not noticeable.(If the prejudice against threads comes from concerns over the GIL -- that also is a misplaced concern in this instance. Since you only have network connection, you will receive only one packet at a time, so only one thread will be active at a time. If the extraction process uses a significant enough amount of CPU time I wonder what that extraction would be, by the way. Unless you ask for compression of the HTTP data, the images come as-is on the TCP stream. so that the extractions are all running at the same time *AND* if you are running on a machine with separate CPU/cores *AND* you would like the extractions to be running truly in parallel on those separate cores, *THEN*, and only then, will processes be more efficient than threads.) I can't remember what the bad advice was, but here processes versus threads clearly doesn't matter performance-wise. I generally recommend processes, because how they work is well-known and they're not as vulnerable to weird synchronization bugs as threads. Second, running 5 wgets is equivalent to 5 processes not 5 threads. And third -- you don't have to use either threads *or* processes. There is another possibility which is much more light-weight: asynchronous I/O, available through the low level select module, or more usefully via the higher-level asyncore module. Yeah, that would be my first choice too for a problem which isn't clearly CPU-bound. Or my second choice -- the first would be calling on a utility like wget(1). /Jorgen -- // Jorgen Grahn grahn@ Oo o. . . \X/ snipabacken.se O o . -- http://mail.python.org/mailman/listinfo/python-list
Re: Do I have to use threads?
Marco Salden wrote: On Jan 6, 5:36 am, Philip Semanchuk phi...@semanchuk.com wrote: On Jan 5, 2010, at 11:26 PM, aditya shukla wrote: Hello people, I have 5 directories corresponding 5 different urls .I want to download images from those urls and place them in the respective directories.I have to extract the contents and download them simultaneously.I can extract the contents and do then one by one. My questions is for doing it simultaneously do I have to use threads? No. You could spawn 5 copies of wget (or curl or a Python program that you've written). Whether or not that will perform better or be easier to code, debug and maintain depends on the other aspects of your program(s). bye Philip Yep, the more easier and straightforward the approach, the better: threads are always (programmers')-error-prone by nature. But my question would be: does it REALLY need to be simultaneously: the CPU/OS only has more overhead doing this in parallel with processess. Measuring sequential processing and then trying to optimize (e.g. for user response or whatever) would be my prefered way to go. Less=More. regards, Marco Threads aren't as hard a some people make out although it does depend on the problem. If your processes are effectively independent then threads are probably the right solution. You can turn any function into a thread quite easily, I posted a function for this a while back... http://groups.google.com/group/comp.lang.python/msg/3361a897db3834b4?dmode=source Also it's often a good idea to build in a flag that switches your app from multi threaded to single threaded as it's easier to debug the latter. Roger. -- http://mail.python.org/mailman/listinfo/python-list
Re: PIL show() not working for 2nd pic
Do you get any errors or warnings? Could we see the code you ran to get this problem? Thanks Sean -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 10:11 AM, Carsten Haese carsten.ha...@gmail.com
mailto:carsten.ha...@gmail.com wrote:
Victor Subervi wrote:
Code snippet:
[...]
Error:
[...]
What do?
After eliminating the pieces of your post that have been copied or
quoted from elsewhere, I am left with a total of five words of your own
to ask this question, which seems to be approximately the same amount of
effort you have put into trying to figure out what's causing the error.
First I get scolded for not including enough information. Now I get
scolded for including too much. Seems I can't win. I *am* putting effort
into understanding this. I'm sorry I'm not as sharp as you are in these
matters.
params = {}, key = 'store', global cgi = module 'cgi' from
'/usr/lib64/python2.4/cgi.pyc'
, cgi.FieldStorage = class cgi.FieldStorage, ].value undefined
TypeError: unsubscriptable object
args = ('unsubscriptable object',)
Why is the value undefined? What is an unsubscriptable object?
TIA,
beno
What is why?
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
--
http://mail.python.org/mailman/listinfo/python-list
Scripting (was Re: Python books, literature etc)
On Thu, 2010-01-07, Peter wrote: [...] depending on your application domain, I liked: 1) Hans Petter Langtangen: Python Scripting for Computational Science A truly excellent book, not only with respect to Python Scripting , but also on how to avoid paying license fees by using opensource tools as an engineer ( plotting, graphing, gui dev etc ). Very good , pratical introduction to Python with careful and non-trivial examples and exercises. Sounds good. Regarding the book's title: is it just me, or are Python programmers in general put off when people call it scripting? I won't attempt a strict definition of the term scripting language, but it seems like non-programmers use it to mean less scary than what you might think of as programming, while programmers interpret it as not useful as a general-purpose language. /Jorgen -- // Jorgen Grahn grahn@ Oo o. . . \X/ snipabacken.se O o . -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote:
[...]
TypeError: unsubscriptable object
args = ('unsubscriptable object',)
Why is the value undefined? What is an unsubscriptable object?
TIA,
beno
Sorry, that wasn't very helpful. Here's some more advice:
Google is you friend. Try Googling for components of the error
traceback. It's unlikely you are the first person to experience this
problem.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
--
http://mail.python.org/mailman/listinfo/python-list
C Module's '1.#INF' changes to 'inf' at Python
Hi All,
My problem is that I've noticed a strange behavior in Python while handling
FPEs on Windows after switching compilers (msvc8 to msvc9) and I am trying to
find out how Python handles INF values to figure out where the problem might
be.
The problem appeared when we noticed that behavior of Numeric (older version of
NumPy) library has changed when dealing with Floating-Point Exceptions. We are
building our own slightly modified version of Python in-house (including
Numeric/NumPy) and when we switched from the msvc8 to msvc9 compiler, Numeric
started to return '-inf' as a result of 'Numeric.log10(0.0)' instead of rasing
an OverflowError. I know that Numeric is using umath instead of the math
library and since math.log10(0.0) is still raising an Error (which is
ValueError in 2.6 and OverflowError in 2.4, but still an error) I thought that
it might have something to do with how umath or Numeric handles the input or
return values of log functions.
Before hunting blindly I wanted to see how Python interprets INF results. I
built a simple module with only one function that causes an FPE with log10(0.0)
using C's math library and prints the results with printf. Then I imported that
module from python, called the function and printed the result in python too to
see if there are any differences between two values (which I found out that
there are)
Here is the that part of the code from my C Python module:
static PyObject *badNum(PyObject *self, PyObject *args)
{
int sts = 0; //Not used, just dummy
double a = 0.0;
double c = 0.0;
if (!PyArg_ParseTuple(args, i, sts)) //Dummy check, no use
return NULL;
disableFPEs();
c = log10(a); //Since FPEs disabled, should return '-1.#INF'
enableFPEs();
printf(C-Val: %f\n, c);
return Py_BuildValue(d, c);
}
After I build the module and imported it into Python, I called the function and
printed the return value in Python. The output was different in two versions of
Python:
Output1 (Module was built with msvc8 and run by python24)
C-Val: -1.#INF
Py-Val: -1.#INF
Output2 (Module was built with msvc9 and run by python26)
C-Val: -1.#INF
Py-Val: -inf
I now know that compiler change didn't cause any change in return value of
log10(0.0) at C level (both compilations resulted in '-1.#INF') so I am
thinking that somehow at somewhere in Python core the value is translated into
'-inf' and that might be the reason why Numeric/NumPy is missing the problem
and having issues raising OverflowError like it's supposed to.
I started debugging Python.exe (with my module loaded) in Visual Studio. While
tracing what's happening after the line return Py_BuildValue(d, c); I went
down to the floatobject.c file (which I thought is highly relevant to
floating-point value conversions) and in particular format_float() function
but since I am not that familiar with Python internals, I am having trouble
pinpointing the issue and the code where the conversion is happening and I
can't even be sure if I am looking at the right place.
I would appreciate if someone could help me out or at least could point me to
the right direction so that I can figure out the rest.
Thanks,
Best Regards,
Ali Erman CELEN
Platform Specialists / Porting
Dassault Systèmes I www.3ds.com http://www.3ds.com/
This email and any attachments are intended solely for the use of the
individual or entity to whom it is addressed and may be confidential and/or
privileged. If you are not one of the named recipients or have received this
email in error, (i) you should not read, disclose, or copy it, (ii) please
notify sender of your receipt by reply email and delete this email and all
attachments, (iii) Dassault Systemes does not accept or assume any liability or
responsibility for any use of or reliance on this email.For other languages, go
to http://www.3ds.com/terms/email-disclaimer.
--
http://mail.python.org/mailman/listinfo/python-list
Re: One function calling another defined in the same file being exec'd
En Thu, 07 Jan 2010 19:47:13 -0300, Mitchell L Model mlm...@comcast.net
escribió:
def dofile(filename):
ldict = {'result': None}
with open(filename) as file:
exec(file.read(), globals(), ldict)
print('Result for {}: {}'.format(filename, ldict['result']))
Next I call dofile() on a slightly more complex file, in which one
function
calls another function defined earlier in the same file.
def fn1(val):
return sum(range(val))
def fn2(arg):
return fn1(arg)
result = fn2(5)
This produces a surprise:
NameError: global name 'fn1' is not defined
Ok - short answer or long answer?
Short answer: Emulate how modules work. Make globals() same as locals().
(BTW, are you sure you want the file to run with the *same* globals as the
caller? It sees the dofile() function and everything you have
defined/imported there...). Simply use: exec(..., ldict, ldict)
[1] How is it that fn2 can be called from the top-level of the script
but fn1
cannot be called from fn2?
Long answer: First, add these lines before result=fn2(5):
print(globals=, globals().keys())
print(locals=, locals().keys())
import dis
dis.dis(fn2)
and you'll get:
globals()= dict_keys(['dofile', '__builtins__', '__file__', '__package__',
'__name__', '__doc__'])
locals()= dict_keys(['result', 'fn1', 'fn2'])
So fn1 and fn2 are defined in the *local* namespace (as always happens in
Python, unless you use the global statement). Now look at the code of fn2:
6 0 LOAD_GLOBAL 0 (fn1)
3 LOAD_FAST0 (arg)
6 CALL_FUNCTION1
9 RETURN_VALUE
Again, the compiler knows that fn1 is not local to fn2, so it must be
global (because there is no other enclosing scope) and emits a LOAD_GLOBAL
instruction. But when the code is executed, 'fn1' is not in the global
scope...
Solution: make 'fn1' exist in the global scope. Since assignments (implied
by the def statement) are always in the local scope, the only alternative
is to make both scopes (global and local) the very same one.
This shows that the identity globals() is locals() is essential for the
module system to work.
[2] Is this correct behavior or is there something wrong with Python
here?
It's perfectly logical once you get it... :)
[3] How should I write a file to be exec'd that defines several
functions that
call each other, as in the trivial fn1-fn2 example above?
Use the same namespace for both locals and globals: exec(file.read(),
ldict, ldict)
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
Re: Recommended new way for config files
On Thu, 2010-01-07, Jean-Michel Pichavant wrote: Peter wrote: Hi There seems to be several strategies to enhance the old ini-style config files with real python code, for example: ... Is there a strategy that should be prefered for new projects ? ... The .ini file is the simpliest solution, at least from the user point of view, no need to learn any python syntax. Yeah. Use whatever your users expect, and deal with it. The language you've happened to implement your stuff in should normally be irrelevant to the users. I wouldn't use .ini-style, but that's because I'm a Unix guy and they remind me of brief and painful experiments with Windows 3.1. Just remember to include support for commented-out lines. /Jorgen -- // Jorgen Grahn grahn@ Oo o. . . \X/ snipabacken.se O o . -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote:
First I get scolded for not including enough information. Now I get
scolded for including too much. Seems I can't win. I *am* putting effort
into understanding this. I'm sorry I'm not as sharp as you are in these
matters.
I didn't scold you for including too much information. I scolded you for
not demonstrating any effort on your part to try to figure out the
problem. Maybe you did put effort into figuring it out, but you didn't
show us any of it. You simply threw your entire code out there and asked
an open-ended question that gave us no insight into your thought process.
params = {}, key = 'store', global cgi = module 'cgi' from
'/usr/lib64/python2.4/cgi.pyc'
, cgi.FieldStorage = class cgi.FieldStorage, ].value undefined
TypeError: unsubscriptable object
args = ('unsubscriptable object',)
Why is the value undefined? What is an unsubscriptable object?
At least those are specific questions. value undefined means that the
cgitb traceback is unable to show a meaningful value for the object
known as cgi.FieldStorage. Maybe you should ask yourself why that is
and why you're operating on an object that doesn't have a meaningful value.
An unsubscriptable object is an object that can't be subscripted. In
other words, it's an object foo for which the expression foo[bar] is
invalid and results in exactly the kind of exception you're struggling
with right now.
-Carsten
--
http://mail.python.org/mailman/listinfo/python-list
Re: Scripting (was Re: Python books, literature etc)
On Fri, Jan 8, 2010 at 09:37, Jorgen Grahn grahn+n...@snipabacken.se wrote: Regarding the book's title: is it just me, or are Python programmers in general put off when people call it scripting? I won't attempt a strict definition of the term scripting language, but it seems like non-programmers use it to mean less scary than what you might think of as programming, while programmers interpret it as not useful as a general-purpose language. I dunno... I consider it programming when I'm writing bash scripts. Same with running python scripts. My personal take on it, so YMMV, is that scripting is just a synonym for programming an interpreted language, as opposed to programming (common parlance) which is usually meant writing code for a compiled language (C, C++, VB, etc...) Then again, I also tend to use scripting, coding and programming interchangeably too. And sometimes scripting = just writing a quick and dirty program to do a small task, programming = writing something much larger for long term use. Either way, I'm not offended by any of those terms as they all involve programming, regardless of whether or not someone actually calls it programming. For another analogy, what do they call Chinese food in China? Food. Cheers Jeff -- Ted Turner - Sports is like a war without the killing. - http://www.brainyquote.com/quotes/authors/t/ted_turner.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Recommended new way for config files
Chris Rebert wrote: On Thu, Jan 7, 2010 at 10:19 AM, Peter vm...@mycircuit.org wrote: snip The .ini file is the simpliest solution, at least from the user point of view, no need to learn any python syntax. I am speaking from the point of view of a python programmer, and I find the .ini restrictions not necessarily simple, for example when dealing with structured data (I suppose it is trivial to specify a dictionnary or a list for the purpose of my request) For example, configuration files for the logging module get unwieldy when you specify several loggers , handlers, formatters etc, because you have to break down structured data ( objects ) to name,value pairs. snip So what is the worshipped approach, when you need more than name=value pairs ? JSON is one option: http://docs.python.org/library/json.html Cheers, Chris -- http://blog.rebertia.com Yes, JSON is rapidly becoming a standard for stuff like this as it's widely portable and less bulky than XML. It's the native format for couchdb too which is nice if you want to replicate and share the contents of your documents. Roger. -- http://mail.python.org/mailman/listinfo/python-list
Re: Accessing python from a network share in windows 7
aj wrote: On Jan 7, 3:51 pm, MRAB pyt...@mrabarnett.plus.com wrote: aj wrote: It works without any issue on win7 if I copy python to my local drive. Also, accessing python from the same network share works fine on win XP. So I am suspecting some security policy of win7 that is causing problem while accessing it over a network share. You might also want to ask on Microsoft Answers... http://social.answers.microsoft.com/Forums/en-US/categories Good luck, Roger -- http://mail.python.org/mailman/listinfo/python-list
Re: C Module's '1.#INF' changes to 'inf' at Python
On 2010-01-08 07:48 AM, CELEN Erman wrote: Hi All, My problem is that I’ve noticed a strange behavior in Python while handling FPEs on Windows after switching compilers (msvc8 to msvc9) and I am trying to find out how Python handles INF values to figure out where the problem might be. The problem appeared when we noticed that behavior of Numeric (older version of NumPy) library has changed when dealing with Floating-Point Exceptions. We are building our own slightly modified version of Python in-house (including Numeric/NumPy) and when we switched from the msvc8 to msvc9 compiler, Numeric started to return ‘-inf’ as a result of ‘Numeric.log10(0.0)’ instead of rasing an OverflowError. I know that Numeric is using umath instead of the math library and since math.log10(0.0) is still raising an Error (which is ValueError in 2.6 and OverflowError in 2.4, but still an error) I thought that it might have something to do with how umath or Numeric handles the input or return values of log functions. Numeric.log10() will check to see if the errno was set to ERANGE. It does not check if a floating point exception flag was set, which is tricky to do across platforms. The newer numpy can do it because we've finally managed to implement all of that platform-specific code, but the earlier Numeric does not. Presumably, the msvc8 C runtime's implementation of log10() sets errno and the msvc9 runtime's version does not. Before hunting blindly I wanted to see how Python interprets INF results. I built a simple module with only one function that causes an FPE with log10(0.0) using C’s math library and prints the results with printf. Then I imported that module from python, called the function and printed the result in python too to see if there are any differences between two values (which I found out that there are) Python 2.6 changed the string representations of the special floating point values inf and nan. Previously, the string representation was left up to the C runtime, so they differed between platforms. Python 2.6 normalized the string representation across all platforms. The underlying values are the same. What version of Python are you using? -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
return params
[snip]
--
http://mail.python.org/mailman/listinfo/python-list
Manipulating pointers in C using ctypes
Hello!
I have to ask a newbie question about manipulating pointers in C using
ctypes. I have a C dll with two functions. The first one creates a
pointer and returns it to python. The second one takes a pointer as an
argument shows its address and the value at which it is pointing. This
I have implemented using the following code
--- pointers.c
#include stdio.h
#ifdef __cplusplus
extern C { // only need to export C interface if
// used by C++ source code
using namespace std;
#endif
__declspec(dllexport) void* create()
{
double number = 2.2;
double* ptr = number;
printf(Pointer address \t %p \n, ptr);
printf(Value at pointer \t %f \n, ptr[0]);
return (void*) ptr;
}
__declspec(dllexport) int show(double* ptr)
{
printf(Pointer address \t %p \n, ptr);
printf(Pointer value \t %f\n, *ptr);
*ptr = 2.4;
printf(New pointer value \t %f\n, *ptr);
return 0;
}
#ifdef __cplusplus
}
#endif
Please note that in the second function, the show function, I want to
manipulate the value at which the pointer points.
Now, I call this function from python with the following script.
--- pointers.py --
import ctypes as ct
# Load dll
pointers = ct.cdll.LoadLibrary('pointers.dll')
getattr(pointers, 'create')
getattr(pointers, 'show')
pointers.create.restype = ct.c_void_p
# Create pointer in C
ptr = pointers.create()
# Show pointer address and value
print 'Adress returned to python ' +hex(ptr)
pointers.show(ct.c_void_p(ptr))
---
Calling this script gives me the following output:
Pointer address 0021E508
Value at pointer 2.20
Adress returned to python 0x21e508
Pointer address 0021E508
Pointer value0.00(2.2 expected)
New pointer value2.40 (2.4 expected)
But the script returns also an error.
WindowsError: exception: access violation reading 0x4003
WARNING: Failure executing file: pointers.py
Another thing that I find strange is that the return value of
pointers.create is actually an integer instead of an ct.c_void_p
object.
Moreover, I also tried to directly manipulate the address of the
pointer given as an argument to pointers.show. But when it returns to
python the pointer points still at the same address as before the
function call.
Can someone help me with this problem? I would be very glad about an
answer.
With kind regards,
Daniel
--
http://mail.python.org/mailman/listinfo/python-list
Advanced Python Programming - March 8-12, 2010
Looking to make the most of Python's advanced features? Then this is the class for you! The Open Technology Group's Advanced Python Programming course focuses on topics of interest to experienced Python programmers, and teaches you how to leverage advanced features and functionality of Python, including: - Python objects, generators, and decorators - Writing and using regular expressions - Using sockets and networking - Improving performance using threads and the multiprocess module - Signal handling - Cooperative multitasking networking with Twisted - Unit testing with the unittest module - XML document processing - and much more! This 5 day course is taught in a hands-on, instructor-led setting with classes limited to no more than 12 students. Hands on exercises allow students to put what they learn into practice immediately, and receive immediate instructor feedback. All-inclusive pricing includes round-trip airfare, hotel accommodation, and local shuttle services for just $2,595 (the course by itself is only $2,295). For more course details and to enroll, visit our web site at http://www.otg-nc.com/advanced-python-training , reply to this message, or contact us at 877-258-8987. Space is limited, so enroll today! -- Chander Ganesan Open Technology Group, Inc. One Copley Parkway, Suite 210 Morrisville, NC 27560 919-463-0999/877-258-8987 http://www.otg-nc.com -- http://mail.python.org/mailman/listinfo/python-list
restructuredText editor ?
What editor do people out there use to edit .rst files for sphinx-python documentation ? Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: please help shrink this each_with_index() implementation
Phlip schreef: Nobody wrote: Writing robust software from the outset puts you at a competitive disadvantage to those who understand how the system works. And I, not my language, should pick and chose how to be rigorous. The language should not make the decision for me. You can always easily make your program less rigorous than the language, but the reverse is generally very difficult. So a rigorous language gives you the choice, why a non-rigorous language does not. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven -- http://mail.python.org/mailman/listinfo/python-list
Re: File transfer with python
On Jan 6, 7:00 pm, Valentin de Pablo Fouce thi...@gmail.com wrote: Hi there, My intention is to transfer files from one computer to another. Hi Valentin, take a look at dropbox! Clients are available for mac, linux and windows. Up and running in just 2 minutes. http://dropbox.com Regards Kamill -- http://mail.python.org/mailman/listinfo/python-list
Re: Scripting (was Re: Python books, literature etc)
Jorgen Grahn wrote: On Thu, 2010-01-07, Peter wrote: [...] depending on your application domain, I liked: 1) Hans Petter Langtangen: Python Scripting for Computational Science A truly excellent book, not only with respect to Python Scripting , but also on how to avoid paying license fees by using opensource tools as an engineer ( plotting, graphing, gui dev etc ). Very good , pratical introduction to Python with careful and non-trivial examples and exercises. Sounds good. Regarding the book's title: is it just me, or are Python programmers in general put off when people call it scripting? I won't attempt a strict definition of the term scripting language, but it seems like non-programmers use it to mean less scary than what you might think of as programming, while programmers interpret it as not useful as a general-purpose language. I'd probably say that in scripting, convenience is more important than speed. You don't need to create a project, just put the code into a file and then run it. -- http://mail.python.org/mailman/listinfo/python-list
Re: restructuredText editor ?
Peter schrieb: What editor do people out there use to edit .rst files for sphinx-python documentation ? Emacs has a RST-mode (which unfortunately has some runtime-issues, the longer the text the edit, the slower it gets), and together with flymake rst2xml, you get nice error-notifications. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Caching objects in a C extension
On Jan 8, 2:59 am, Diez B. Roggisch de...@nospam.web.de wrote:
casevh schrieb:
I'm working with a C extension that needs to rapidly create and delete
objects. I came up with an approach to cache objects that are being
deleted and resurrect them instead of creating new objects. It appears
to work well but I'm afraid I may be missing something (besides
heeding the warning in the documentation that _Py_NewReference is for
internal interpreter use only).
Below is a simplified version of the approach I'm using:
MyType_dealloc(MyTypeObject *self)
{
if(I_want_to_save_MyType(self)) {
// Save the object pointer in a cache
save_it(self);
} else {
PyObject_Del(self);
}
}
MyType_new(void)
{
MyTypeObject *self;
if(there_is_an_object_in_the_cache) {
self = get_object_from_cache;
_Py_NewReference((PyObject*)self);
} else {
if(!(self = PyObjectNew(MyTypeObject, MyType))
return NULL;
initialize_the_new_object(self);
}
return self;
}
The objects referenced in the cache have a reference count of 0 and I
don't increment the reference count until I need to resurrect the
object. Could these objects be clobbered by the garbage collector?
Would it be safer to create the new reference before stuffing the
object into the cache (even though it will look like there is a memory
leak when running under a debug build)?
Deep out of my guts I'd say keeping a reference, and using you own
LRU-scheme would be the safest without residing to use dark magic.
Diez- Hide quoted text -
- Show quoted text -
Thanks for the reply. I realized that I missed one detail. The objects
are created by the extension but are deleted by Python. I don't know
that an object is no longer needed until its tp_dealloc is called. At
that point, its reference count is 0.
casevh
--
http://mail.python.org/mailman/listinfo/python-list
Re: Caching objects in a C extension
Le Fri, 08 Jan 2010 08:39:17 -0800, casevh a écrit : Thanks for the reply. I realized that I missed one detail. The objects are created by the extension but are deleted by Python. I don't know that an object is no longer needed until its tp_dealloc is called. At that point, its reference count is 0. tuple objects and others already have such a caching scheme, so you could download the Python source and look at e.g. Objects/tupleobject.c to see how it's done. Regards Antoine. -- http://mail.python.org/mailman/listinfo/python-list
Re: One function calling another defined in the same file being exec'd
On Jan 7, 2010, at 10:45 PM, Steven D'Aprano st...@remove-this-cybersource.com.au wrote an extensive answer to my questions about one function calling another in the same file being exec'd. His suggestion about printing out locals() and globals() in the various possible places provided the clues to explain what was going on. I would like to summarize what I have learned from this, because although I have known all the relevant pieces for many years I never put them together in a way that explains the odd behavior I observed. Statements that bind new names -- assignment, def, and class -- do so in the local scope. While exec'ing a file the local scope is determined by the arguments passed to exec; in my case, I passed an explicit local scope. It was particularly obtuse of me not to notice the effects of this because I was intentionally using it so that an assignment to 'result' in the exec'd script would enable the exec'ing code to retrieve the value of result. However, although the purity of Python with respect to the binding actions of def and class statements is wonderful and powerful, it is very difficult cognitively to view a def on a page and think aha! that's just like an assignment of a newly created function to a name, even though that is precisely the documented behavior of def. So mentally I was making an incorrect distinction between what was getting bound locally and what was getting bound globally in the exec'd script. Moreover, the normal behavior of imported code, in which any function in the module can refer to any other function in the module, seduced me into this inappropriate distinction. To my eye I was just defining and using function definitions the way they are in modules. There is a key difference between module import and exec: as Steven pointed out, inside a module locals() is globals(). On further reflection, I will add that what appears to be happening is that during import both the global and local dictionaries are set to a copy of the globals() from the importing scope and that copy becomes the value of the module's __dict__ once import has completed successfully. Top-level statements bind names in locals(), as always, but because locals() and globals() are the same dictionary, they are also binding them in globals(), so that every function defined in the module uses the modified copy of globals -- the value of the module's __dict__ -- as its globals() when it executes. Because exec leaves locals() and globals() distinct, functions defined at the top level of a string being exec'd don't see other assignments and definitions that are also in the string. Another misleading detail is that top-level expressions in the exec can use other top-level names (assigned, def'd, etc.), which they will find in the exec string's local scope, but function bodies do not see the string's local scope. The problem I encountered arises because the function definitions need to access each other through the global scope, not the local scope. In fact, the problem would arise if one of the functions tried to call itself recursively, since its own name would not be in the global scope. So we have a combination of two distinctions: the different ways module import and exec use globals and locals and the difference between top-level statements finding other top-level names in locals but functions looking for them in globals. Sorry for the long post. These distinctions go deep into the semantics of Python namespaces, which though they are lean, pure, and beautiful, have some consequences that can be surprising -- more so the more familiar you are with other languages that do things differently. Oh, and as far as using import instead of exec for my scripts, I don't think that's appropriate, if only because I don't want my application's namespace polluted by what could be many of these pseudo- modules users might load during a session. (Yes, I could remove the name once the import is finished, but importing solely for side- effects rather than to use the imported module is offensive. Well, I would be using one module name -- result -- but that doesn't seem to justify the complexities of setting up the import and accessing the module when exec does in principle just what I need.) Finally, once all of this is really understood, there is a simple way to change an exec string's def's to bind globally instead of locally: simply begin the exec with a global declaration for any function called by one of the others. In my example, adding a global fn1 at the beginning of the file fixes it so exec works. global fn1# enable fn1 to be called from fn2! def fn1(val): return sum(range(val)) def fn2(arg): return fn1(arg) result = fn2(5) -- http://mail.python.org/mailman/listinfo/python-list
Re: One function calling another defined in the same file being exec'd
On Jan 8, 2010, at 9:55 AM, Gabriel Genellina gagsl- p...@yahoo.com.ar wrote: Ok - short answer or long answer? Short answer: Emulate how modules work. Make globals() same as locals(). (BTW, are you sure you want the file to run with the *same* globals as the caller? It sees the dofile() function and everything you have defined/imported there...). Simply use: exec(..., ldict, ldict) [1] How is it that fn2 can be called from the top-level of the script but fn1 cannot be called from fn2? Long answer: First, add these lines before result=fn2(5): print(globals=, globals().keys()) print(locals=, locals().keys()) import dis dis.dis(fn2) and you'll get: globals()= dict_keys(['dofile', '__builtins__', '__file__', '__package__', '__name__', '__doc__']) locals()= dict_keys(['result', 'fn1', 'fn2']) So fn1 and fn2 are defined in the *local* namespace (as always happens in Python, unless you use the global statement). Now look at the code of fn2: 6 0 LOAD_GLOBAL 0 (fn1) 3 LOAD_FAST0 (arg) 6 CALL_FUNCTION1 9 RETURN_VALUE Again, the compiler knows that fn1 is not local to fn2, so it must be global (because there is no other enclosing scope) and emits a LOAD_GLOBAL instruction. But when the code is executed, 'fn1' is not in the global scope... Solution: make 'fn1' exist in the global scope. Since assignments (implied by the def statement) are always in the local scope, the only alternative is to make both scopes (global and local) the very same one. This is very helpful additional information and clarification! Thanks. This shows that the identity globals() is locals() is essential for the module system to work. Yes, though I doubt more than a few Python programmers would guess that identity. [2] Is this correct behavior or is there something wrong with Python here? It's perfectly logical once you get it... :) I think I'm convinced. [3] How should I write a file to be exec'd that defines several functions that call each other, as in the trivial fn1-fn2 example above? Use the same namespace for both locals and globals: exec(file.read(), ldict, ldict) I was going to say that this wouldn't work because the script couldn't use any built-in names, but the way exec works if the value passed for the globals argument doesn't contain an entry for '__builtins__' it adds one. I would have a further problem in that there are some names I want users to be able to use in their scripts, in particular classes that have been imported into the scope of the code doing the exec, but come to think of it I don't want to expose the entire globals() anyway. The solution is do use the same dictionary for both globals and locals, as you suggest, to emulate the behavior of module import, and explicitly add to it the names I want to make available (and since they are primarily classes, there are relatively few of those, as opposed to an API of hundreds of functions). Thanks for the help. -- http://mail.python.org/mailman/listinfo/python-list
Re: PyQt QThreadPool error
On 8 янв, 16:27, Phil Thompson p...@riverbankcomputing.com wrote: On Thu, 7 Jan 2010 15:07:10 -0800 (PST), h0uk vardan.pogos...@gmail.com wrote: ... Phil you right about app.exec_(). But situation is sligthly different. I want to have more than one Job. I add these Jobs into QThreadPool trough cycle. And I also want these Jobs to run sequentially. The following code illustrate what I mean: #!/usr/bin/env python # -*- coding: utf-8 -*- import sys import os import time from PyQt4 import QtCore, QtGui class Job(QtCore.QRunnable): def __init__(self, name): QtCore.QRunnable.__init__(self) self._name = name def run(self): time.sleep(3) print self._name if __name__ == __main__: app = QtGui.QApplication(sys.argv) QtCore.QThreadPool.globalInstance().setMaxThreadCount(1) for i in range(5): j = Job(Job- + str(i)) j.setAutoDelete(True) QtCore.QThreadPool.globalInstance().start(j, i) app.exec_() After 5 cycle I get the same error: An unhandled win32 exception occured in python.exe. How I can do it?? To run my Jobs sequentially??? It's a PyQt bug. The workaround is not to use setAutoDelete() and instead keep an explicit reference to your Job instances - for example in a list of jobs. Phil Thanks, Phil. -- http://mail.python.org/mailman/listinfo/python-list
Re: Caching objects in a C extension
casevh schrieb:
On Jan 8, 2:59 am, Diez B. Roggisch de...@nospam.web.de wrote:
casevh schrieb:
I'm working with a C extension that needs to rapidly create and delete
objects. I came up with an approach to cache objects that are being
deleted and resurrect them instead of creating new objects. It appears
to work well but I'm afraid I may be missing something (besides
heeding the warning in the documentation that _Py_NewReference is for
internal interpreter use only).
Below is a simplified version of the approach I'm using:
MyType_dealloc(MyTypeObject *self)
{
if(I_want_to_save_MyType(self)) {
// Save the object pointer in a cache
save_it(self);
} else {
PyObject_Del(self);
}
}
MyType_new(void)
{
MyTypeObject *self;
if(there_is_an_object_in_the_cache) {
self = get_object_from_cache;
_Py_NewReference((PyObject*)self);
} else {
if(!(self = PyObjectNew(MyTypeObject, MyType))
return NULL;
initialize_the_new_object(self);
}
return self;
}
The objects referenced in the cache have a reference count of 0 and I
don't increment the reference count until I need to resurrect the
object. Could these objects be clobbered by the garbage collector?
Would it be safer to create the new reference before stuffing the
object into the cache (even though it will look like there is a memory
leak when running under a debug build)?
Deep out of my guts I'd say keeping a reference, and using you own
LRU-scheme would be the safest without residing to use dark magic.
Diez- Hide quoted text -
- Show quoted text -
Thanks for the reply. I realized that I missed one detail. The objects
are created by the extension but are deleted by Python. I don't know
that an object is no longer needed until its tp_dealloc is called. At
that point, its reference count is 0.
I don't fully understand. Whoever creates these objects, you get a
reference to them at some point. Then you increment (through the
destined Macros) the ref-count.
All objects in your pool with refcount 1 are canditates for removal. All
you need to do is to keep a kind of timestamp together with them, since
when they are released. If that's to old, fully release them.
Diez
--
http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
return params
[snip]
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/
Holden Web LLC http://www.holdenweb.com/
UPCOMING EVENTS:http://holdenweb.eventbrite.com/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Caching objects in a C extension
On Jan 8, 9:19 am, Diez B. Roggisch de...@nospam.web.de wrote:
casevh schrieb:
On Jan 8, 2:59 am, Diez B. Roggisch de...@nospam.web.de wrote:
casevh schrieb:
I'm working with a C extension that needs to rapidly create and delete
objects. I came up with an approach to cache objects that are being
deleted and resurrect them instead of creating new objects. It appears
to work well but I'm afraid I may be missing something (besides
heeding the warning in the documentation that _Py_NewReference is for
internal interpreter use only).
Below is a simplified version of the approach I'm using:
MyType_dealloc(MyTypeObject *self)
{
if(I_want_to_save_MyType(self)) {
// Save the object pointer in a cache
save_it(self);
} else {
PyObject_Del(self);
}
}
MyType_new(void)
{
MyTypeObject *self;
if(there_is_an_object_in_the_cache) {
self = get_object_from_cache;
_Py_NewReference((PyObject*)self);
} else {
if(!(self = PyObjectNew(MyTypeObject, MyType))
return NULL;
initialize_the_new_object(self);
}
return self;
}
The objects referenced in the cache have a reference count of 0 and I
don't increment the reference count until I need to resurrect the
object. Could these objects be clobbered by the garbage collector?
Would it be safer to create the new reference before stuffing the
object into the cache (even though it will look like there is a memory
leak when running under a debug build)?
Deep out of my guts I'd say keeping a reference, and using you own
LRU-scheme would be the safest without residing to use dark magic.
Diez- Hide quoted text -
- Show quoted text -
Thanks for the reply. I realized that I missed one detail. The objects
are created by the extension but are deleted by Python. I don't know
that an object is no longer needed until its tp_dealloc is called. At
that point, its reference count is 0.
I don't fully understand. Whoever creates these objects, you get a
reference to them at some point. Then you increment (through the
destined Macros) the ref-count.
All objects in your pool with refcount 1 are canditates for removal. All
you need to do is to keep a kind of timestamp together with them, since
when they are released. If that's to old, fully release them.
Diez- Hide quoted text -
- Show quoted text -
These are numeric objects created by gmpy. I'm trying to minimize the
overhead for using mpz with small numbers. Objects are created and
deleted very often by the interpreter as expressions are evaluated. I
don't keep ownership of the objects.
casevh
--
http://mail.python.org/mailman/listinfo/python-list
monitor reading file with thread
I'm reading and processing a huge file, so during the execution I want to now the state of the processing: how many lines are already processed, and so on. The first approach is: f = open(filename) n = 0 for l in f: if n % 1000 = 0: print Reading %d lines %n do_something(l) but I want something diffent. I want to monitor the running of the computing every n seconds. It's the first time I use threading, and I think mine is not the best solution: import threading import time import Queue class Reader(): def __init__(self,filename): self.filename = filename self.lineno = 0 def __iter__(self): f = open(self.filename) for line in f: self.lineno += 1 time.sleep(0.01) # slow down yield line f.close() class Monitor(threading.Thread): def __init__(self,reader,stop_queue,interval=2): threading.Thread.__init__(self) self.interval = interval self.reader = reader self.stop_queue = stop_queue def run(self): while True: try: if self.stop_queue.get(timeout=self.interval) == stop: break except Queue.Empty: pass print MONITOR: , reader.lineno reader = Reader(r1_200910.log) q = Queue.Queue() monitor = Monitor(reader,q) monitor.start() for line in reader: pass # do_somethinghard(line) q.put(stop) monitor.join() It't works, but I don't like how I'm stopping the thread. In general the Monitor class can be more complex, for example a progress bar. Using python 2.6 -- http://mail.python.org/mailman/listinfo/python-list
table from csv file
I am trying to create a table in python from a csv file where I input
which columns I would like to see, and the table only shows those
columns. I have attached an example of the csv file i am using, and
some of the code I have written. I am having trouble converting
variables between lists, dictionaries and tuples. Is there a name for
what I am attempting to do? any help to get me on the right track with
this is appreciated.
test.csv
DateOpen HighLowClose Volume Adj Close
12/14/09110.01 110.7 109.5 110.24 16316000110.24
12/11/09110.6 110.82 108.72 109.32 28983100109.32
12/10/09110.66 111.03 110.04 110.82 20491700110.82
12/9/09 111.6 112.48 109.38 110.84 37104900110.84
12/8/09 112.76 113.06 110.21 110.95 37630800110.95
12/7/09 111.51 114.22 111.44 113.11 42480500113.11
12/4/09 117.15 117.26 112.41 113.75 79182600113.75
12/3/09 118.57 119.54 118.03 118.7 28802100118.7
12/2/09 118.8 119.27 118.3 119.18 30994600119.18
12/1/09 117.3 117.93 116.78 117.38 27641000117.38
11/30/09114.48 115.89 114.27 115.64 16402300115.64
11/27/09113.08 115.81 113.02 115.06 21234400115.06
11/25/09115.69 116.88 115.53 116.62 24553300116.62
11/24/09114.73 114.81 113.97 114.73 22599700114.73
11/23/09114.67 115.12 113.99 114.29 24422700114.29
11/20/09111.74 112.94 111.54 112.94 17302500112.94
11/19/09111.85 112.4 110.76 112.3 21239800112.3
11/18/09112.69 113.09 111.8 112.25 22320600112.25
11/17/09111.09 111.99 110.9 111.97 19732900111.97
11/16/09110.7 112.16 110.65 111.63 25002300111.63
11/13/09108.32 109.8 108.14 109.74 17246000109.74
11/12/09109.16 109.56 108.12 108.21 17848300108.21
11/11/09109.49 109.71 109 109.6 17654100109.6
11/10/09108.03 108.78 107.7 108.39 15973300108.39
11/9/09 108.69 108.75 107.91 108.19 18444800108.19
11/6/09 107.38 108.04 107.06 107.43 14789000107.43
11/5/09 106.81 107.2 106.6 106.98 10189000106.98
11/4/09 107.11 107.68 106.43 107.1 27125500107.1
table.py;
import csv
(D, O, H, L, C, V, A) = (11, 'open', 'high', 'low', 'close', 66, 77)
d = {'high':H, 'low':L, 'close':C, 'open':O}
spacing = '%-*s'
w=raw_input('what do you want to see? (use commas to seperate values)
')
y=w.lower()
x=y.replace(' ','')
print x
p = x.split(',')#this takes string, converts to
list
print p
num = len(p)
format = num*spacing
width = 12*num
wi = 12
secwidth = width - wi
bb = []
i=0
while i num: #creates new list with variables
ll = d[p[i]]
bb.insert(i,ll)
i+=1
print bb
i = 0
while i num:
bb.insert(i*2, 12) #this works on list
i+=1
print bb
i = 0
while i num:
p.insert(i*2, 12) #this works on list
i+=1
q = tuple (p) #takes list, converts to tuple
reader = csv.reader(open('/prog/test.csv', rb))
rownum = 0
for row in reader:
if rownum == 0:
print '=' * width
print format % (q)
print '-' * width
print ''
else:
D, O, H, L, C, V, A = row [:7]
o = tuple (bb)
print format % (o)
rownum += 1
--
http://mail.python.org/mailman/listinfo/python-list
Re: Need help to pass self.count to other classes.
Steve Holden wrote: Steven D'Aprano wrote: [... points out my misapprehension ...] kbi = kbInterface() sys.ps1 = kbi.prompt1 bound method kbInterface.prompt1 of __main__.kbInterface object at 0xb7cbd52cprint Hello Hello bound method kbInterface.prompt1 of __main__.kbInterface object at 0xb7cbd52c Right, this is expert mode ... Here's a way to turn expert mode into something less advanced: import sys def expert_mode(): ... return for newbies ;) ... sys.ps1 = expert_mode function expert_mode at 0x7f498b9735f0class Str: ... def __init__(self, f): self.f = f ... def __str__(self): return self.f() ... function expert_mode at 0x7f498b9735f0sys.ps1 = Str(expert_mode) for newbies ;) Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic text color
On Wed, Jan 6, 2010 at 9:18 AM, John Posner jjpos...@optimum.net wrote: On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick mackrac...@gmail.com wrote: But it is not what I am wanting. I first thought to make it look for a space but that would not work when a single character like # is to be colored if there is a string of them. Or if all of the characters between quotes are to be colored. Regular expressions are good at handling searches like: * all the characters between quotes * the two-character string do, but only if it's a complete word -John -- http://mail.python.org/mailman/listinfo/python-list I need another hint... Been doing some reading and playing and it looks like r'\bxxx\b' is what I need. But I can not figure out how to pass a variable between \b___\b If the word in question is between the \b \b and in the list then it works like I want it to. The below does not work. greenList_regexp = |.join(greenList) for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): start,end = matchobj.span() Thanks, Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden st...@holdenweb.com wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I exactly copied from
a Web page tutorial. Can you folks point me to tutorials where I can learn
to comprehend this code? Specifically, the line in question. How is it that
one can code params[key] (what does that mean?) and the other side, what
does that mean
--
http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden st...@holdenweb.com
mailto:st...@holdenweb.com wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I exactly copied
from a Web page tutorial. Can you folks point me to tutorials where I
can learn to comprehend this code? Specifically, the line in question.
How is it that one can code params[key] (what does that mean?) and
the other side, what does that mean
I think you are gathering more fans Victor :)
http://docs.python.org/tutorial/datastructures.html#dictionaries
JM
--
http://mail.python.org/mailman/listinfo/python-list
Re: Recommended new way for config files
On Jan 7, 8:12 pm, Peter vm...@mycircuit.org wrote: snip So what is the worshipped approach, when you need more than name=value pairs ? JSON is one option:http://docs.python.org/library/json.html Thanks, didn't think about that, although most of the apps I know don't seem to use this approach for improved conf file handling ( ipython, pylons, etc ). To me , the ipython way ( hybrid: ipy_user_conf.py and *.ini files ) seems to be the most comprehensive way amongst the larger apps I know of, since it let you have a python coded file for what ever you might want to do during initialization and have additional .ini files, ,possibily several in different locations, for simpler options in name,value format. Has anybody experiences with other tools that use this approach ? Peter I don't know if you will find it relevant to your needs, but PEP 391 suggests an alternative method of configuring logging using dicts. The use of dicts as the basic format (rather than Python code) allows for some end-user flexibility and interoperability with other systems: e.g. JSON, YAML and Python can all produce Python dicts. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: table from csv file
On Jan 8, 5:59 pm, marlowe marlowequ...@hotmail.com wrote:
I am trying to create a table in python from a csv file where I input
which columns I would like to see, and the table only shows those
columns. I have attached an example of the csv file i am using, and
some of the code I have written. I am having trouble converting
variables between lists, dictionaries and tuples. Is there a name for
what I am attempting to do? any help to get me on the right track with
this is appreciated.
test.csv
Date Open High Low Close Volume Adj Close
12/14/09 110.01 110.7 109.5 110.24 16316000 110.24
12/11/09 110.6 110.82 108.72 109.32 28983100 109.32
12/10/09 110.66 111.03 110.04 110.82 20491700 110.82
12/9/09 111.6 112.48 109.38 110.84 37104900 110.84
12/8/09 112.76 113.06 110.21 110.95 37630800 110.95
12/7/09 111.51 114.22 111.44 113.11 42480500 113.11
12/4/09 117.15 117.26 112.41 113.75 79182600 113.75
12/3/09 118.57 119.54 118.03 118.7 28802100 118.7
12/2/09 118.8 119.27 118.3 119.18 30994600 119.18
12/1/09 117.3 117.93 116.78 117.38 27641000 117.38
11/30/09 114.48 115.89 114.27 115.64 16402300 115.64
11/27/09 113.08 115.81 113.02 115.06 21234400 115.06
11/25/09 115.69 116.88 115.53 116.62 24553300 116.62
11/24/09 114.73 114.81 113.97 114.73 22599700 114.73
11/23/09 114.67 115.12 113.99 114.29 24422700 114.29
11/20/09 111.74 112.94 111.54 112.94 17302500 112.94
11/19/09 111.85 112.4 110.76 112.3 21239800 112.3
11/18/09 112.69 113.09 111.8 112.25 22320600 112.25
11/17/09 111.09 111.99 110.9 111.97 19732900 111.97
11/16/09 110.7 112.16 110.65 111.63 25002300 111.63
11/13/09 108.32 109.8 108.14 109.74 17246000 109.74
11/12/09 109.16 109.56 108.12 108.21 17848300 108.21
11/11/09 109.49 109.71 109 109.6 17654100 109.6
11/10/09 108.03 108.78 107.7 108.39 15973300 108.39
11/9/09 108.69 108.75 107.91 108.19 18444800 108.19
11/6/09 107.38 108.04 107.06 107.43 14789000 107.43
11/5/09 106.81 107.2 106.6 106.98 10189000 106.98
11/4/09 107.11 107.68 106.43 107.1 27125500 107.1
table.py;
import csv
(D, O, H, L, C, V, A) = (11, 'open', 'high', 'low', 'close', 66, 77)
d = {'high':H, 'low':L, 'close':C, 'open':O}
spacing = '%-*s'
w=raw_input('what do you want to see? (use commas to seperate values)
')
y=w.lower()
x=y.replace(' ','')
print x
p = x.split(',') #this takes string, converts to
list
print p
num = len(p)
format = num*spacing
width = 12*num
wi = 12
secwidth = width - wi
bb = []
i=0
while i num: #creates new list with variables
ll = d[p[i]]
bb.insert(i,ll)
i+=1
print bb
i = 0
while i num:
bb.insert(i*2, 12) #this works on list
i+=1
print bb
i = 0
while i num:
p.insert(i*2, 12) #this works on list
i+=1
q = tuple (p) #takes list, converts to tuple
reader = csv.reader(open('/prog/test.csv', rb))
rownum = 0
for row in reader:
if rownum == 0:
print '=' * width
print format % (q)
print '-' * width
print ''
else:
D, O, H, L, C, V, A = row [:7]
o = tuple (bb)
print format % (o)
rownum += 1
This might be a useful starting point (I'm guessing this is what
you're after...)
Let's assume your 'CSV' file is tab separated as it's certainly not
comma separated :)
import csv
csvin = csv.reader(open('test.csv'), delimiter='\t')
header = dict( (val.strip(),idx) for idx, val in enumerate(next
(csvin)) )
We can use header as a column name-column index lookup eg header
['Open'] == 1
from operator import itemgetter
wanted = ['Open', 'Close'] # Although you'll want to use raw_input and
split on ','
getcols = itemgetter(*[header[col] for col in wanted])
getcols is a helper function that'll return a tuple of the columns in
the requested order...
for row in csvin:
print getcols(row)
Loop over the rest of the file and output the required columns.
hth
Jon.
--
http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant
jeanmic...@sequans.com wrote:
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden st...@holdenweb.commailto:
st...@holdenweb.com wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I exactly copied
from a Web page tutorial. Can you folks point me to tutorials where I can
learn to comprehend this code? Specifically, the line in question. How is it
that one can code params[key] (what does that mean?) and the other side,
what does that mean
I think you are gathering more fans Victor :)
Fans? You aught to hear me sing. I didn't think I'd gain fans here LOL!
http://docs.python.org/tutorial/datastructures.html#dictionaries
Thanks,
beno
--
http://mail.python.org/mailman/listinfo/python-list
Re: Manipulating pointers in C using ctypes
On Jan 8, 7:37 am, Daniel Platz mail.to.daniel.pl...@googlemail.com
wrote:
Hello!
I have to ask a newbie question about manipulating pointers in C using
ctypes. I have a C dll with two functions. The first one creates a
pointer and returns it to python. The second one takes a pointer as an
argument shows its address and the value at which it is pointing. This
I have implemented using the following code
--- pointers.c
#include stdio.h
#ifdef __cplusplus
extern C { // only need to export C interface if
// used by C++ source code
using namespace std;
#endif
__declspec(dllexport) void* create()
{
double number = 2.2;
double* ptr = number;
printf(Pointer address \t %p \n, ptr);
printf(Value at pointer \t %f \n, ptr[0]);
return (void*) ptr;
}
__declspec(dllexport) int show(double* ptr)
{
printf(Pointer address \t %p \n, ptr);
printf(Pointer value \t %f\n, *ptr);
*ptr = 2.4;
printf(New pointer value \t %f\n, *ptr);
return 0;
}
#ifdef __cplusplus}
#endif
Please note that in the second function, the show function, I want to
manipulate the value at which the pointer points.
Now, I call this function from python with the following script.
--- pointers.py --
import ctypes as ct
# Load dll
pointers = ct.cdll.LoadLibrary('pointers.dll')
getattr(pointers, 'create')
getattr(pointers, 'show')
pointers.create.restype = ct.c_void_p
# Create pointer in C
ptr = pointers.create()
# Show pointer address and value
print 'Adress returned to python ' +hex(ptr)
pointers.show(ct.c_void_p(ptr))
---
Calling this script gives me the following output:
Pointer address 0021E508
Value at pointer 2.20
Adress returned to python 0x21e508
Pointer address 0021E508
Pointer value 0.00 (2.2 expected)
New pointer value 2.40 (2.4 expected)
But the script returns also an error.
WindowsError: exception: access violation reading 0x4003
WARNING: Failure executing file: pointers.py
Another thing that I find strange is that the return value of
pointers.create is actually an integer instead of an ct.c_void_p
object.
Moreover, I also tried to directly manipulate the address of the
pointer given as an argument to pointers.show. But when it returns to
python the pointer points still at the same address as before the
function call.
Can someone help me with this problem? I would be very glad about an
answer.
With kind regards,
Daniel
try this:
__declspec(dllexport) void* create()
{
double* ptr = new double;
*ptr = 2.2;
printf(Pointer address \t %p \n, ptr);
printf(Value at pointer \t %f \n, ptr[0]);
return (void*) ptr;
}
Basically once you leave create() the address being used for number is
no longer valid. The other implication for this is you are going to
need a delete() function to free that memory you allocated for the
double as well.
--
http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant
jeanmic...@sequans.com wrote:
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden st...@holdenweb.commailto:
st...@holdenweb.com wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I exactly copied
from a Web page tutorial. Can you folks point me to tutorials where I can
learn to comprehend this code? Specifically, the line in question. How is it
that one can code params[key] (what does that mean?) and the other side,
what does that mean
I think you are gathering more fans Victor :)
http://docs.python.org/tutorial/datastructures.html#dictionaries
This still isn't telling me what I need to know. Perhaps I'm missing the
point, as in the recent case with putting the print cookie statement in
the header. I am assuming (translation: making an a$$ out of you and me)
that params[key] automatically assigns the fieldStorage key to the newly
created params key in the dict of the same, and assigning the value from the
cgi.FieldStorage of that key to the params value. Is that correct?
TIA,
beno
--
http://mail.python.org/mailman/listinfo/python-list
Mencoder not working from a python script
Hello guys, I am trying to create a python script which uses mencoder to create videos from a set of images.When I am using the command(windows 7) i am able to generate the video properly.But when used within a python script i am ggetting error.Below is the code and error.any help is appreciated. import *subprocess*,*sys*,os def *createVideo*(): mencoder = *C:\\MPlayer-p4-svn-30075\\mencoder.exe* path=*mf://\C:\\videos\\test\\*.jpg\ * output=*\C:\\videos\\test\\xyz.avi\* commande = *%s %s -mf w=800:h=600:fps=25:type=jpg -ovc copy -oac copy -o %s* os.system(commande % (mencoder, path,output)) if __name__==*__main__*: createVideo() error- MEncoder Sherpya-SVN-r30075-4.2.5 (C) 2000-2009 MPlayer Team success: format: 16 data: 0x0 - 0x0 MF file format detected. [mf] search expr: C:\cameras\Perkins Road\*.jpg [mf] number of files: 301 (1204) VIDEO: [IJPG] 800x600 24bpp 25.000 fps 0.0 kbps ( 0.0 kbyte/s) [V] filefmt:16 fourcc:0x47504A49 size:800x600 fps:25.000 ftime:=0.0400 videocodec: framecopy (800x600 24bpp fourcc=47504a49) Writing index... Writing header... ODML: Aspect information not (yet?) available or unspecified, not writing vprp header. Video stream: nan kbit/s (-2147483648 B/s) size: 0 bytes 0.000 secs 0 frames Thanks Aditya -- http://mail.python.org/mailman/listinfo/python-list
lightweight encryption of text file
I have a plain text file which I would like to protect in a very simple minded, yet for my purposes sufficient, way. I'd like to encrypt/convert it into a binary file in such a way that possession of a password allows anyone to convert it back into the original text file while not possessing the password one would only see the following with the standard linux utility 'file': [fetchin...@fetch ~]$ file encrypted.data encrypted.data: data and the effort required to convert the file back to the original text file without the password would be equivalent to guessing the password. I'm fully aware of the security implications of this loose specification, but for my purposes this would be a good solution. What would be the simplest way to achieve this using preferably stock python without 3rd party modules? If a not too complex 3rd party module made it really simple that would be acceptable too. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi victorsube...@gmail.comwrote:
On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant
jeanmic...@sequans.com wrote:
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden st...@holdenweb.commailto:
st...@holdenweb.com wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I exactly copied
from a Web page tutorial. Can you folks point me to tutorials where I can
learn to comprehend this code? Specifically, the line in question. How is it
that one can code params[key] (what does that mean?) and the other side,
what does that mean
I think you are gathering more fans Victor :)
http://docs.python.org/tutorial/datastructures.html#dictionaries
This still isn't telling me what I need to know. Perhaps I'm missing the
point, as in the recent case with putting the print cookie statement in
the header. I am assuming (translation: making an a$$ out of you and me)
that params[key] automatically assigns the fieldStorage key to the newly
created params key in the dict of the same, and assigning the value from the
cgi.FieldStorage of that key to the params value. Is that correct?
TIA,
beno
I may have answered my own question. I have this:
def cgiFieldStorageToDict(fieldStorage):
params = {}
for key in fieldStorage.keys():
params[key] = fieldStorage[key].value
return params
dict = cgiFieldStorageToDict(cgi.FieldStorage())
print dict
which gave me this:
{'store': 'products', 'cat': 'prodCat1'}
which looks about right. I would have written the code like this:
keys = []
values = []
for key, value in fieldStorage.iteritems():
keys.append(key)
values.append(value)
params = dict(zip(keys, values))
which obviously isn't as elegant. But that's what I knew. Learned another
trick, I guess ;)
Thanks all.
beno
--
http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden st...@holdenweb.com
mailto:st...@holdenweb.com wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I exactly copied
from a Web page tutorial. Can you folks point me to tutorials where I
can learn to comprehend this code? Specifically, the line in question.
How is it that one can code params[key] (what does that mean?) and the
other side, what does that mean
If you got it from:
Recipe 81547: Using a simple dictionary for CGI parameters
http://code.activestate.com/recipes/81547/
then no, it wasn't exactly copied. If you'd copy-and-pasted it then it
would've been correct, as well as much quicker to do...
--
http://mail.python.org/mailman/listinfo/python-list
Re: Scripting (was Re: Python books, literature etc)
Sounds good. Regarding the book's title: is it just me, or are Python programmers in general put off when people call it scripting? I won't attempt a strict definition of the term scripting language, but it seems like non-programmers use it to mean less scary than what you might think of as programming, while programmers interpret it as not useful as a general-purpose language. It took me a while to take scripting seriously. I grew up with Pascal and Eiffel and I found it difficult to appreciate dynamic typing and scripting. The author Langtangen is explaining in detail why he considers scripting useful, in particular he provides an automatic test suite to run different language versions ( perl, python, c, c++) of the same program to compare performance. The results are amazing, in that some of the examples run faster than the C++ version. I find Python extremly useful as a general purpose language ( its clearly now my prefered one ) and I find it equally useful to develop toy apps in C++, Haskell and Lisp, just to better appreciate the idea of general purpose. For me, it has turned out that the point is not scripting versus not scripting or static versus dynamic typing but having automatic unittests or not having automatic unittests. My most important module is nose for running unittests the easy way. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic text color
Dave McCormick wrote: On Wed, Jan 6, 2010 at 9:18 AM, John Posner jjpos...@optimum.net mailto:jjpos...@optimum.net wrote: On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick mackrac...@gmail.com mailto:mackrac...@gmail.com wrote: But it is not what I am wanting. I first thought to make it look for a space but that would not work when a single character like # is to be colored if there is a string of them. Or if all of the characters between quotes are to be colored. Regular expressions are good at handling searches like: * all the characters between quotes * the two-character string do, but only if it's a complete word -John -- http://mail.python.org/mailman/listinfo/python-list I need another hint... Been doing some reading and playing and it looks like r'\bxxx\b' is what I need. But I can not figure out how to pass a variable between \b___\b If the word in question is between the \b \b and in the list then it works like I want it to. The below does not work. greenList_regexp = |.join(greenList) for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): start,end = matchobj.span() The regex r'\bgreenList_regexp\b' will match the string 'greenList_regexp' if it's a whole word. What you mean is any of these words, provided that they're whole words. You'll need to group the alternatives within (?:...), like this: r'\b(?:' + greenList_regexp + ')\b' -- http://mail.python.org/mailman/listinfo/python-list
Re: Recommended new way for config files
On 01/08/2010 03:57 PM, r0g wrote: Chris Rebert wrote: On Thu, Jan 7, 2010 at 10:19 AM, Petervm...@mycircuit.org wrote: snip The .ini file is the simpliest solution, at least from the user point of view, no need to learn any python syntax. I am speaking from the point of view of a python programmer, and I find the .ini restrictions not necessarily simple, for example when dealing with structured data (I suppose it is trivial to specify a dictionnary or a list for the purpose of my request) For example, configuration files for the logging module get unwieldy when you specify several loggers , handlers, formatters etc, because you have to break down structured data ( objects ) to name,value pairs. snip So what is the worshipped approach, when you need more than name=value pairs ? JSON is one option: http://docs.python.org/library/json.html Cheers, Chris -- http://blog.rebertia.com Yes, JSON is rapidly becoming a standard for stuff like this as it's widely portable and less bulky than XML. It's the native format for couchdb too which is nice if you want to replicate and share the contents of your documents. Roger. Excellent, thanks , have to check that out. Any experience with yaml http://yaml.org/ ? Peter -- http://mail.python.org/mailman/listinfo/python-list
Another Screwy Problem
Hi;
I have this line of code:
sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid)
which prints to this:
select Name, Price from productsPackages where ID=1;
which when I enter it into the MySQL interpreter gives me this:
mysql select Name, Price from productsPackages where ID=1;
+--++
| Name | Price |
+--++
| pkg | 123.45 |
+--++
1 row in set (0.00 sec)
exactly what I expect. However, in my script for some reason it returns
this:
((1,),)
Why would it do that? I guess there's some foolish thing I did in my code
somewhere, but I'll be darned if I know where. Here's the whole script:
#! /usr/bin/python
import cgitb; cgitb.enable()
import cgi
import MySQLdb
import sys,os
sys.path.append(os.getcwd())
from login import login
from template import top, bottom
from sets import Set
import fpformat
form = cgi.FieldStorage()
store = form.getfirst('store')
cat = form.getfirst('cat', '')
user, passwd, db, host = login()
db = MySQLdb.connect(host, user, passwd, db)
cursor = db.cursor()
def displayProducts(patientID=''):
try: # These are stores with categories where ordering by price is
important
sql = 'select ID from %s where Category=%s order by Price desc;' %
(store, cat)
cursor.execute(sql)
except: # Stores, like prescriptions, where ordering by price is not
important
sql = 'select ID from %s;' % (store)
cursor.execute(sql)
ids = [itm[0] for itm in cursor]
cursor.execute('describe %s;' % store)
colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for itm in
cursor]
i = 0
if len(ids) 0:
for id in ids:
# print 'tr\n'
print 'form method=post action=displayOneProduct.py'
print input type='hidden' name='store' value='%s' / % store
print input type='hidden' name='id' value='%s' / % id
j = 0
for col in colFields:
sql = 'select %s from %s where ID=%s;' % (col, store, str(id))
cursor.execute(sql)
colValue = cursor.fetchone()
if col == 'SKU':
print input type='hidden' name='sku' value='%s' / %
colValue[0]
print 'b%s: /b%sbr /\n' % (col, colValue[0])
elif col == 'Category':
print input type='hidden' name='cat' value='%s' / %
colValue[0]
print 'b%s: /b%sbr /\n' % (col, colValue[0])
elif col == 'OutOfStock':
if colValue[0] == '1': # This product is out of stock
outOfStockFlag = 'yes'
else:
outOfStockFlag = 'no'
elif col[:3] != 'pic':
notSet = 1
if isinstance(colValue[0], (str, int, float, long, complex,
unicode, list, buffer, xrange, tuple)):
pass
else:
try:
html = b%s/b: select name='%s' % (col, col)
notSet = 0
for itm in colValue[0]:
try:
color, number = itm.split('$')
html += option name='%s'%s/option % (itm, color)
except:
html += option name='%s'%s/option % (itm, itm)
html += /selectbr /
print html
except:
pass
if notSet == 1:
if len(col) 49:
colX = col[:50] + '...'
else:
colX = col
print 'b%s: /b%sbr /\n' % (colX, colValue[0])
elif col == 'pic1':
try:
if (colValue[0] != None):
#if (colValue[0] != None) and (len(colValue[0] 0)):
print 'a href=getpic.py?store=%spic=%sid=%s
class=highslide href=getpic.py?store=%spic=%sid=%s onclick=return
hs.expand(this)img src=getpic.py?store=%spic=%sid=%s width=100
height=80 alt= align=left border=0 title=Click to enlarge
style=border: 0px/abr clear=all /\n' % (store, col[3:], id, store,
col[3:], id, store, col[3:], id)
except TypeError:
raise
except:
raise
# i += 1
# try:
#content = colValues[0][x].tostring()
#pic = tmp + str(i) + .jpg
#try:
# os.remove(pic)
#except:
# pass
#img = open(pic, w)
#img.write(content)
#print 'img src=%sbr /br /' % pic
#img.close()
# except:
#pass
j += 1
if store != 'prescriptions':
if outOfStockFlag == 'yes':
print 'font color=redThis item is currently bout of
stock/b./font'
else:
print input type='submit' value=' More Info ' /
else:
print input type='hidden' name='patientID' value='%s' / %
patientID
print input type='submit' value=' More Info ' /
print '/formbr /br /'
print '/td/tr/table\n'
print '/table\n'
def cgiFieldStorageToDict(fieldStorage):
params = {}
for key in fieldStorage.keys():
params[key] = fieldStorage[key].value
return params
def display():
top()
dict = cgiFieldStorageToDict(cgi.FieldStorage())
print
Re: getfirst and re
On Fri, Jan 8, 2010 at 3:21 PM, MRAB pyt...@mrabarnett.plus.com wrote:
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden st...@holdenweb.commailto:
st...@holdenweb.com wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I exactly copied
from a Web page tutorial. Can you folks point me to tutorials where I can
learn to comprehend this code? Specifically, the line in question. How is it
that one can code params[key] (what does that mean?) and the other side,
what does that mean
If you got it from:
Recipe 81547: Using a simple dictionary for CGI parameters
http://code.activestate.com/recipes/81547/
then no, it wasn't exactly copied. If you'd copy-and-pasted it then it
would've been correct, as well as much quicker to do...
I'm afraid to say you're right again :-}
beno
--
http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi victorsube...@gmail.com
mailto:victorsube...@gmail.com wrote:
On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant
jeanmic...@sequans.com mailto:jeanmic...@sequans.com wrote:
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden
st...@holdenweb.com mailto:st...@holdenweb.com
mailto:st...@holdenweb.com mailto:st...@holdenweb.com
wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I
exactly copied from a Web page tutorial. Can you folks point
me to tutorials where I can learn to comprehend this code?
Specifically, the line in question. How is it that one can
code params[key] (what does that mean?) and the other
side, what does that mean
I think you are gathering more fans Victor :)
http://docs.python.org/tutorial/datastructures.html#dictionaries
This still isn't telling me what I need to know. Perhaps I'm missing
the point, as in the recent case with putting the print cookie
statement in the header. I am assuming (translation: making an a$$
out of you and me) that params[key] automatically assigns the
fieldStorage key to the newly created params key in the dict of the
same, and assigning the value from the cgi.FieldStorage of that key
to the params value. Is that correct?
TIA,
beno
I may have answered my own question. I have this:
def cgiFieldStorageToDict(fieldStorage):
params = {}
for key in fieldStorage.keys():
params[key] = fieldStorage[key].value
return params
Which is what Recipe 81547 actually says!
dict = cgiFieldStorageToDict(cgi.FieldStorage())
print dict
which gave me this:
{'store': 'products', 'cat': 'prodCat1'}
which looks about right. I would have written the code like this:
keys = []
values = []
for key, value in fieldStorage.iteritems():
keys.append(key)
values.append(value)
params = dict(zip(keys, values))
which obviously isn't as elegant. But that's what I knew. Learned
another trick, I guess ;)
If that works then so should this:
params = dict(fieldStorage.iteritems())
--
http://mail.python.org/mailman/listinfo/python-list
Re: lightweight encryption of text file
On 2010-01-08 13:14 PM, Daniel Fetchinson wrote: I have a plain text file which I would like to protect in a very simple minded, yet for my purposes sufficient, way. I'd like to encrypt/convert it into a binary file in such a way that possession of a password allows anyone to convert it back into the original text file while not possessing the password one would only see the following with the standard linux utility 'file': [fetchin...@fetch ~]$ file encrypted.data encrypted.data: data and the effort required to convert the file back to the original text file without the password would be equivalent to guessing the password. I'm fully aware of the security implications of this loose specification, but for my purposes this would be a good solution. What would be the simplest way to achieve this using preferably stock python without 3rd party modules? If a not too complex 3rd party module made it really simple that would be acceptable too. Paul Rubin's p3.py algorithm is probably the most straightforward way to meet these requirements. It's not a standard crypto algorithm by any means, but Paul knows his stuff and has devised it with these deployment restrictions in mind. http://www.nightsong.com/phr/crypto/p3.py -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic text color
On Fri, Jan 8, 2010 at 12:28 PM, MRAB pyt...@mrabarnett.plus.com wrote: Dave McCormick wrote: On Wed, Jan 6, 2010 at 9:18 AM, John Posner jjpos...@optimum.netmailto: jjpos...@optimum.net wrote: On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick mackrac...@gmail.com mailto:mackrac...@gmail.com wrote: But it is not what I am wanting. I first thought to make it look for a space but that would not work when a single character like # is to be colored if there is a string of them. Or if all of the characters between quotes are to be colored. Regular expressions are good at handling searches like: * all the characters between quotes * the two-character string do, but only if it's a complete word -John --http://mail.python.org/mailman/listinfo/python-list I need another hint... Been doing some reading and playing and it looks like r'\bxxx\b' is what I need. But I can not figure out how to pass a variable between \b___\b If the word in question is between the \b \b and in the list then it works like I want it to. The below does not work. greenList_regexp = |.join(greenList) for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): start,end = matchobj.span() The regex r'\bgreenList_regexp\b' will match the string 'greenList_regexp' if it's a whole word. What you mean is any of these words, provided that they're whole words. You'll need to group the alternatives within (?:...), like this: r'\b(?:' + greenList_regexp + ')\b' Thanks but that did not work either. Maybe I have something else wrong too? complete_text = Tbox.get(1.0, END) greenList = green grass.split() greenList_regexp = |.join(greenList) for matchobj in re.finditer(r'\b(?:' + greenList_regexp + ')\b', complete_text): start,end = matchobj.span() Tbox.tag_add(green, 1.0 + %d chars % start,1.0 + %d chars % end) Tbox.tag_config(green, foreground=green) The words green and grass do not turn green. Dave -- http://mail.python.org/mailman/listinfo/python-list
Re: getfirst and re
On Fri, Jan 8, 2010 at 3:37 PM, MRAB pyt...@mrabarnett.plus.com wrote:
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 3:09 PM, Victor Subervi
victorsube...@gmail.commailto:
victorsube...@gmail.com wrote:
On Fri, Jan 8, 2010 at 2:52 PM, Jean-Michel Pichavant
jeanmic...@sequans.com mailto:jeanmic...@sequans.com wrote:
Victor Subervi wrote:
On Fri, Jan 8, 2010 at 1:26 PM, Steve Holden
st...@holdenweb.com mailto:st...@holdenweb.com
mailto:st...@holdenweb.com mailto:st...@holdenweb.com
wrote:
MRAB wrote:
Victor Subervi wrote:
[snip]
Code snippet:
def cgiFieldStorageToDict(fieldStorage):
Further hint ...
params = {}
for key in fieldStorage.keys():
params[key] = cgi.FieldStorage[key].value
^
This is your problem.
The problem is that I don't understand this code that I
exactly copied from a Web page tutorial. Can you folks point
me to tutorials where I can learn to comprehend this code?
Specifically, the line in question. How is it that one can
code params[key] (what does that mean?) and the other
side, what does that mean
I think you are gathering more fans Victor :)
http://docs.python.org/tutorial/datastructures.html#dictionaries
This still isn't telling me what I need to know. Perhaps I'm missing
the point, as in the recent case with putting the print cookie
statement in the header. I am assuming (translation: making an a$$
out of you and me) that params[key] automatically assigns the
fieldStorage key to the newly created params key in the dict of the
same, and assigning the value from the cgi.FieldStorage of that key
to the params value. Is that correct?
TIA,
beno
I may have answered my own question. I have this:
def cgiFieldStorageToDict(fieldStorage):
params = {}
for key in fieldStorage.keys():
params[key] = fieldStorage[key].value
return params
Which is what Recipe 81547 actually says!
dict = cgiFieldStorageToDict(cgi.FieldStorage())
print dict
which gave me this:
{'store': 'products', 'cat': 'prodCat1'}
which looks about right. I would have written the code like this:
keys = []
values = []
for key, value in fieldStorage.iteritems():
keys.append(key)
values.append(value)
params = dict(zip(keys, values))
which obviously isn't as elegant. But that's what I knew. Learned another
trick, I guess ;)
If that works then so should this:
params = dict(fieldStorage.iteritems())
Yeah. Thanks!
beno
--
http://mail.python.org/mailman/listinfo/python-list
Re: asyncore based port splitter code questions
On 4 Gen, 18:58, George Trojan george.tro...@noaa.gov wrote:
asyncore based code is supposed to be simple,
but I need while loops and a lot of try/except clauses.
First of all: you DON'T have to use while loops or anything which is
blocking where by blocking I mean anything like time.sleep().
asyncore, just like Twisted, is an asynchrounous abstraction on top of
select().
Anything you call must return *immediately* otherwise the whole thing
will hang aka stop serving any connected client or server.
I designed the code by looking at Python 2.3
source for asyncore and originally wanted to use add_channel() and
del_channel() methods. However in Python 2.6 del_channel() closes the
socket in addition to deleting it from the map.
Don't look at the 2.3 source. Use asyncore of Python 2.6 which is far
more improved, bug-fixed and also *different*, where by that I mean
that it might actually behaves differently.
If you are forced to use Python 2.3 my advice is to get a copy of
Python's asyncore.py and asynchat.py and include them in your code.
Secondly, to temporarily sleep your connections *don't* remove
anything from your map.
The correct way of doing things here is to override readable() and
writable() methods and make them return False as long as you want your
connection to hang.
Now I'm going to comment some parts of your code.
class Reader(asyncore.dispatcher):
def __init__(self, sock, writers):
asyncore.dispatcher.__init__(self, sock)
self.writers = writers
def handle_read(self):
data = self.recv(1024)
for writer in self.writers:
writer.add_data(data)
[...]
def handle_write(self):
while self.data:
log_msg('sending data to %s' % str(self.address))
sent = self.send(self.data)
self.data = self.data[sent:]
self.suspend_channel()
By looking at how you are appending data you want to send in a buffer,
it looks like you might want to use asynchat.async_chat rather than
asyncore.dispatcher.
async_chat.push() and async_chat.push_with_producer() methods already
take care of buffers logic and make sure that all the data gets sent
to the other peer without going lost.
Actually there's no reason to use asyncore.dispatcher class directly
except for creating a socket which listens on an interface and then
passes the connection to another class inheriting from
asynchat.async_chat which will actually handle that session.
So my recommendation is to use asynchat.async_chat whenever possible.
def suspend_channel(self, map=None):
fd = self._fileno
if map is None:
map = self._map
if fd in map:
del map[fd]
As I said this is unecessary.
Override readable() and writable() methods instead.
def handle_close(self):
log_msg('closing writer connection')
self.close()
# try to reconnect
time.sleep(TMOUT)
self.mksocket()
You really don't want to use time.sleep() there.
It blocks everything.
while True:
try:
disp = Dispatcher(port, destinations)
asyncore.loop(timeout=TMOUT, use_poll=True)
except socket.error, (errno, e):
if errno == 98:
log_msg('sleeping %d s: %s', (30, e))
time.sleep(30)
Same as above.
As a final note I would recommend to take a look at pyftpdlib code
which uses asyncore/asynchat as part of its core:
http://code.google.com/p/pyftpdlib
It can be of some help to figure out how things should be done.
--- Giampaolo
http://code.google.com/p/pyftpdlib/
--
http://mail.python.org/mailman/listinfo/python-list
Re: PIL show() not working for 2nd pic
On Jan 8, 6:32 am, McColgst mccol...@gmail.com wrote: Do you get any errors or warnings? Could we see the code you ran to get this problem? Thanks Sean I used to get no such file or directory showing some files in /tmp directory. But today I am getting a different type of message, which appeared yesterday also. (eog:8368): Gtk-CRITICAL **: gtk_tree_model_get_iter: assertion `path- depth 0' failed (eog:8368): Gtk-CRITICAL **: gtk_list_store_get_value: assertion `VALID_ITER (iter, list_store)' failed (eog:8368): GLib-GObject-WARNING **: /build/buildd/glib2.0-2.20.1/ gobject/gtype.c:3940: type id `0' is invalid (eog:8368): GLib-GObject-WARNING **: can't peek value table for type `invalid' which is not currently referenced Segmentation fault ---code-- #!/usr/bin/python print Aum Amriteshwaryai Namaha import Image imagePath = /home/suresh/EE241/book_images_3ed/ch03/ im34 = Image.open(imagePath + breast_digital_Xray.tif) im35 = Image.open(imagePath + DFT_no_log.tif) im35.show() def neg(x): return 255-1-x import math def logtr(x): y = math.log(1+x,10) print y return y*100 im34x = im34.point(neg) im34x.show() im35x = im35.point(logtr) im35x.show() ---end of code Any ideas? Thanks suresh -- http://mail.python.org/mailman/listinfo/python-list
Re: table from csv file
On Fri, Jan 8, 2010 at 13:55, Jon Clements jon...@googlemail.com wrote:
On Jan 8, 5:59 pm, marlowe marlowequ...@hotmail.com wrote:
I am trying to create a table in python from a csv file where I input
which columns I would like to see, and the table only shows those
columns. I have attached an example of the csv file i am using, and
some of the code I have written. I am having trouble converting
variables between lists, dictionaries and tuples. Is there a name for
what I am attempting to do? any help to get me on the right track with
this is appreciated.
test.csv
I had to edit that and comma delimit it, because cut and paste gave me
random numbers/types of whitespace...
[code snipped]
This might be a useful starting point (I'm guessing this is what
you're after...)
Let's assume your 'CSV' file is tab separated as it's certainly not
comma separated :)
import csv
csvin = csv.reader(open('test.csv'), delimiter='\t')
header = dict( (val.strip(),idx) for idx, val in enumerate(next
(csvin)) )
We can use header as a column name-column index lookup eg header
['Open'] == 1
from operator import itemgetter
wanted = ['Open', 'Close'] # Although you'll want to use raw_input and
split on ','
getcols = itemgetter(*[header[col] for col in wanted])
getcols is a helper function that'll return a tuple of the columns in
the requested order...
for row in csvin:
print getcols(row)
Loop over the rest of the file and output the required columns.
As someone who knows just enough to be dangerous... what about this:
import csv
reader = open('C:/test.txt','rb')
data = csv.DictReader(reader,restval='000',restkey='Misc')
print Options are: Date, Open, Close, High, Low, Volume, Adj Close
column = raw_input('What do you want? (comma delimited)?')
choice = column.split(',')
f = 12
print ''.join([s.center(f) for s in choice])
for row in data:
print ''.join([(row.get(s)).center(f) for s in choice])
--
Mike Ditka - If God had wanted man to play soccer, he wouldn't have
given us arms. -
http://www.brainyquote.com/quotes/authors/m/mike_ditka.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: Another Screwy Problem
Victor Subervi wrote: Hi; I have this line of code: sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid) which prints to this: select Name, Price from productsPackages where ID=1; which when I enter it into the MySQL interpreter gives me this: mysql select Name, Price from productsPackages where ID=1; +--++ | Name | Price | +--++ | pkg | 123.45 | +--++ 1 row in set (0.00 sec) exactly what I expect. However, in my script for some reason it returns this: ((1,),) The only logical explanation is that this is not the output from executing the above-mentioned query. Maybe you should ask yourself what it actually is. -Carsten -- http://mail.python.org/mailman/listinfo/python-list
24 bit signed integer binary conversion help needed
hi; I am trying to read 24bit signed WAV format (little endian) data from a WAV file and convert it to 32 bit little endian integer format ... can anybody please tell me how to do the conversion from 24 bit to 32 bit with a snippet of Python code ??? Thanks so much Robert Somerville -- http://mail.python.org/mailman/listinfo/python-list
Re: Threading change, 2.5.4 - 2.6.1
anon wrote: Gib Bogle wrote: The code below runs with Python 2.5.4, but gives the following error messages with Python 2.6.1. What needs to be done to make it work? Thanks. C:\Summer09\Tutorialspython url_queue.pyw Traceback (most recent call last): File url_queue.pyw, line 3, in module import threading File C:\Summer09\Tutorials\threading.py, line 9, in module class ProcessingThread(threading.Thread, QtCore.QObject): AttributeError: 'module' object has no attribute 'Thread' Nothing to do with the version of python You have a file called threading.py in C:\Summer09\Tutorials (the same folder as url_queue.pyw). The line 'import threading' is finding this module before python's threading module. Rename this file and you should be fine. Keep in mind that if there's a threading.pyc in C:\Summer09\Tutorials, you have to rename (or delete) that one as well. Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: 24 bit signed integer binary conversion help needed
On 2010-01-08, Robert Somerville rsomervi...@sjgeophysics.com wrote: I am trying to read 24bit signed WAV format (little endian) data from a WAV file and convert it to 32 bit little endian integer format ... can anybody please tell me how to do the conversion from 24 bit to 32 bit with a snippet of Python code ??? def sext24(d): if ord(d[2]) 0x80: return d+'\xff' else: return d+'\x00' -- Grant Edwards grante Yow! Am I having fun yet? at visi.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic text color
On Fri, 08 Jan 2010 14:28:57 -0500, MRAB pyt...@mrabarnett.plus.com wrote: Dave McCormick wrote: On Wed, Jan 6, 2010 at 9:18 AM, John Posner jjpos...@optimum.net mailto:jjpos...@optimum.net wrote: On Tue, 05 Jan 2010 16:54:44 -0500, Dave McCormick mackrac...@gmail.com mailto:mackrac...@gmail.com wrote: But it is not what I am wanting. I first thought to make it look for a space but that would not work when a single character like # is to be colored if there is a string of them. Or if all of the characters between quotes are to be colored. Regular expressions are good at handling searches like: * all the characters between quotes * the two-character string do, but only if it's a complete word -John -- http://mail.python.org/mailman/listinfo/python-list I need another hint... Been doing some reading and playing and it looks like r'\bxxx\b' is what I need. But I can not figure out how to pass a variable between \b___\b If the word in question is between the \b \b and in the list then it works like I want it to. The below does not work. greenList_regexp = |.join(greenList) for matchobj in re.finditer(r'\bgreenList_regexp\b', complete_text): start,end = matchobj.span() The regex r'\bgreenList_regexp\b' will match the string 'greenList_regexp' if it's a whole word. What you mean is any of these words, provided that they're whole words. You'll need to group the alternatives within (?:...), like this: r'\b(?:' + greenList_regexp + ')\b' Oops, MRAB, you forgot to make the last literal a RAW string -- it should be r')\b' Dave, we're already into some pretty heavy regular-expression work, huh?. Here's another approach -- not nearly as elegant as MRAB's: Given this list: greenList = ['green', 'grass', 'grump'] ... you currently are using join() to construct this regexp search string: 'green|grass|grump' ... but you've decided that you really want this similar regexp search string: r'\bgreen\b|\bgrass\b|\bgrump\b' You can achieve this by transforming each item on the list, then invoking join() on the transformed list to create the search string. Here are a couple of ways to transform the list: * List comprehension: whole_word_greenList = [ r'\b' + word + r'\b' for word in greenList] * map() and a user-defined function: def xform_to_wholeword_searchstring(word): return r'\b' + word + r'\b' whole_word_greenList = map(xform_to_wholeword_searchstring, greenList) HTH, John -- http://mail.python.org/mailman/listinfo/python-list
Re: 24 bit signed integer binary conversion help needed
On 8-1-2010 22:12, Robert Somerville wrote: hi; I am trying to read 24bit signed WAV format (little endian) data from a WAV file and convert it to 32 bit little endian integer format ... can anybody please tell me how to do the conversion from 24 bit to 32 bit with a snippet of Python code ??? Thanks so much Robert Somerville Are you using the standard wave module? I guess that will produce a string of 3-byte audio frames with readframes(). Won't it work to chop this up in individual 3-byte frames, then appending a '\0' char to every frame, and then writing the frames to the target wave ? -irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: 24 bit signed integer binary conversion help needed
On 2010-01-08, Grant Edwards inva...@invalid.invalid wrote:
On 2010-01-08, Robert Somerville rsomervi...@sjgeophysics.com wrote:
I am trying to read 24bit signed WAV format (little endian) data from a
WAV file and convert it to 32 bit little endian integer format ... can
anybody please tell me how to do the conversion from 24 bit to 32 bit
with a snippet of Python code ???
def sext24(d):
if ord(d[2]) 0x80:
return d+'\xff'
else:
return d+'\x00'
I guess I assumed you knew how to read 3 bytes of data from a
file:
f = open('datafile','rb')
d = f.read(3)
--
Grant Edwards grante Yow! I need to discuss
at BUY-BACK PROVISIONS
visi.comwith at least six studio
SLEAZEBALLS!!
--
http://mail.python.org/mailman/listinfo/python-list
Re: lightweight encryption of text file
I have a plain text file which I would like to protect in a very simple minded, yet for my purposes sufficient, way. I'd like to encrypt/convert it into a binary file in such a way that possession of a password allows anyone to convert it back into the original text file while not possessing the password one would only see the following with the standard linux utility 'file': [fetchin...@fetch ~]$ file encrypted.data encrypted.data: data and the effort required to convert the file back to the original text file without the password would be equivalent to guessing the password. I'm fully aware of the security implications of this loose specification, but for my purposes this would be a good solution. What would be the simplest way to achieve this using preferably stock python without 3rd party modules? If a not too complex 3rd party module made it really simple that would be acceptable too. Paul Rubin's p3.py algorithm is probably the most straightforward way to meet these requirements. It's not a standard crypto algorithm by any means, but Paul knows his stuff and has devised it with these deployment restrictions in mind. http://www.nightsong.com/phr/crypto/p3.py Thanks a lot, currently I'm having trouble using this code on python 2.6 but probably some small tweaking will fix it. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown -- http://mail.python.org/mailman/listinfo/python-list
Re: 24 bit signed integer binary conversion help needed
On 2010-01-08, Irmen de Jong irmen-nosp...@xs4all.nl wrote: On 8-1-2010 22:12, Robert Somerville wrote: hi; I am trying to read 24bit signed WAV format (little endian) data from a WAV file and convert it to 32 bit little endian integer format ... can anybody please tell me how to do the conversion from 24 bit to 32 bit with a snippet of Python code ??? Are you using the standard wave module? I guess that will produce a string of 3-byte audio frames with readframes(). Won't it work to chop this up in individual 3-byte frames, then appending a '\0' char to every frame, and then writing the frames to the target wave ? Not if the 3-byte values are signed two's compliment values. -- Grant Edwards grante Yow! I feel like I am at sharing a ``CORN-DOG'' visi.comwith NIKITA KHRUSCHEV ... -- http://mail.python.org/mailman/listinfo/python-list
PIL how to display multiple images side by side
Hi, In PIL, how to display multiple images in say m rows and n colums when I have m*n images. suresh -- http://mail.python.org/mailman/listinfo/python-list
Re: lightweight encryption of text file
I have a plain text file which I would like to protect in a very
simple minded, yet for my purposes sufficient, way. I'd like to
encrypt/convert it into a binary file in such a way that possession of
a password allows anyone to convert it back into the original text
file while not possessing the password one would only see the
following with the standard linux utility 'file':
[fetchin...@fetch ~]$ file encrypted.data
encrypted.data: data
and the effort required to convert the file back to the original text
file without the password would be equivalent to guessing the
password.
I'm fully aware of the security implications of this loose
specification, but for my purposes this would be a good solution.
What would be the simplest way to achieve this using preferably stock
python without 3rd party modules? If a not too complex 3rd party
module made it really simple that would be acceptable too.
Paul Rubin's p3.py algorithm is probably the most straightforward way to
meet
these requirements. It's not a standard crypto algorithm by any means,
but
Paul
knows his stuff and has devised it with these deployment restrictions in
mind.
http://www.nightsong.com/phr/crypto/p3.py
Thanks a lot, currently I'm having trouble using this code on python
2.6 but probably some small tweaking will fix it.
Actually, it also doesn't work with python 2.5 and currently I don't
have access to anything older. array.array raises a
ValueError: string length not a multiple of item size
Does anyone recall a change to array.array?
The full traceback is
Traceback (most recent call last):
File p3.py, line 163, in module
_test()
File p3.py, line 143, in _test
c1 = e(plain,key)
File p3.py, line 69, in p3_encrypt
xkey = _expand_key(k_enc, n+4)
File p3.py, line 41, in _expand_key
return array ('L', j)
ValueError: string length not a multiple of item size
Cheers,
Daniel
--
Psss, psss, put it down! - http://www.cafepress.com/putitdown
--
http://mail.python.org/mailman/listinfo/python-list
Re: 24 bit signed integer binary conversion help needed
On 8-1-2010 22:37, Grant Edwards wrote: On 2010-01-08, Irmen de Jongirmen-nosp...@xs4all.nl wrote: Are you using the standard wave module? I guess that will produce a string of 3-byte audio frames with readframes(). Won't it work to chop this up in individual 3-byte frames, then appending a '\0' char to every frame, and then writing the frames to the target wave ? Not if the 3-byte values are signed two's compliment values. Meh. The wave module is pretty useless for format conversion then until it grows some utility methods. -irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: Another Screwy Problem
Victor Subervi wrote:
Hi;
I have this line of code:
sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid)
which prints to this:
select Name, Price from productsPackages where ID=1;
which when I enter it into the MySQL interpreter gives me this:
mysql select Name, Price from productsPackages where ID=1;
+--++
| Name | Price |
+--++
| pkg | 123.45 |
+--++
1 row in set (0.00 sec)
exactly what I expect. However, in my script for some reason it returns
this:
((1,),)
First, never use string formatting to pass parameters to your database. Read
the MySQLdb documentation (or sqlite, or psycopg2) documentation for reasons
why, and how to do it right.
Second, in the same documentation, look up anything having the word fetch in
it. That should show you how to get the data you want
Third, please be more specific in your questions. You say In my script for
some reason *it* returns this: ((1,),), but you don't show us what it is.
How did you get that from your script? So far we've got a variable called sql
with your query in it. How do I get the same wrong result you got? I don't
know. I could pass it to a function that looks like this:
def botch_sql_query(sql):
return ((1,),)
I could furthermore fix it, so that it looks right, by doing this:
def fixed_sql_query(sql):
return(('pkg', 123.45),)
But that probably doesn't help. Give us enough code to be clear, but not so
much as to be overwhelming. There's an essay called How to ask smart
question by Eric Raymond that should help.
Cheers,
Cliff
--
http://mail.python.org/mailman/listinfo/python-list
Re: Another Screwy Problem
On Fri, Jan 08, 2010 at 03:32:34PM -0400, Victor Subervi wrote regarding
Another Screwy Problem:
Date: Fri, 8 Jan 2010 15:32:34 -0400
From: Victor Subervi victorsube...@gmail.com
To: python-list python-list@python.org
Subject: Another Screwy Problem
Hi;
I have this line of code:
sql = 'select Name, Price from %sPackages where ID=%s;' % (store, pid)
which prints to this:
select Name, Price from productsPackages where ID=1;
which when I enter it into the MySQL interpreter gives me this:
mysql select Name, Price from productsPackages where ID=1;
+--++
| Name | Price |
+--++
| pkg | 123.45 |
+--++
1 row in set (0.00 sec)
exactly what I expect. However, in my script for some reason it returns
this:
((1,),)
Why would it do that? I guess there's some foolish thing I did in my
code somewhere, but I'll be darned if I know where. Here's the whole
script:
#! /usr/bin/python
import cgitb; cgitb.enable()
import cgi
import MySQLdb
import sys,os
sys.path.append(os.getcwd())
from login import login
from template import top, bottom
from sets import Set
import fpformat
form = cgi.FieldStorage()
store = form.getfirst('store')
cat = form.getfirst('cat', '')
user, passwd, db, host = login()
db = MySQLdb.connect(host, user, passwd, db)
cursor = db.cursor()
def displayProducts(patientID=''):
try: # These are stores with categories where ordering by price is
important
sql = 'select ID from %s where Category=%s order by Price desc;'
% (store, cat)
cursor.execute(sql)
except: # Stores, like prescriptions, where ordering by price is not
important
sql = 'select ID from %s;' % (store)
cursor.execute(sql)
ids = [itm[0] for itm in cursor]
cursor.execute('describe %s;' % store)
colFields, colFieldValues = [itm[0] for itm in cursor], [itm[1] for
itm in cursor]
i = 0
if len(ids) 0:
for id in ids:
# print 'tr\n'
print 'form method=post action=displayOneProduct.py'
print input type='hidden' name='store' value='%s' / % store
print input type='hidden' name='id' value='%s' / % id
j = 0
for col in colFields:
sql = 'select %s from %s where ID=%s;' % (col, store,
str(id))
cursor.execute(sql)
colValue = cursor.fetchone()
if col == 'SKU':
print input type='hidden' name='sku' value='%s' / %
colValue[0]
print 'b%s: /b%sbr /\n' % (col, colValue[0])
elif col == 'Category':
print input type='hidden' name='cat' value='%s' / %
colValue[0]
print 'b%s: /b%sbr /\n' % (col, colValue[0])
elif col == 'OutOfStock':
if colValue[0] == '1': # This product is out of stock
outOfStockFlag = 'yes'
else:
outOfStockFlag = 'no'
elif col[:3] != 'pic':
notSet = 1
if isinstance(colValue[0], (str, int, float, long, complex,
unicode, list, buffer, xrange, tuple)):
pass
else:
try:
html = b%s/b: select name='%s' % (col, col)
notSet = 0
for itm in colValue[0]:
try:
color, number = itm.split('$')
html += option name='%s'%s/option % (itm,
color)
except:
html += option name='%s'%s/option % (itm, itm)
html += /selectbr /
print html
except:
pass
if notSet == 1:
if len(col) 49:
colX = col[:50] + '...'
else:
colX = col
print 'b%s: /b%sbr /\n' % (colX, colValue[0])
elif col == 'pic1':
try:
if (colValue[0] != None):
#if (colValue[0] != None) and (len(colValue[0] 0)):
print 'a href=getpic.py?store=%spic=%sid=%s
class=highslide href=getpic.py?store=%spic=%sid=%s
onclick=return hs.expand(this)img
src=getpic.py?store=%spic=%sid=%s width=100 height=80 alt=
align=left border=0 title=Click to enlarge style=border:
0px/abr clear=all /\n' % (store, col[3:], id, store, col[3:],
id, store, col[3:], id)
except TypeError:
raise
except:
raise
# i += 1
# try:
#content = colValues[0][x].tostring()
#pic = tmp + str(i) + .jpg
#try:
# os.remove(pic)
#except:
# pass
#img = open(pic, w)
#img.write(content)
#print 'img src=%sbr
Re: lightweight encryption of text file
On 8-1-2010 22:39, Daniel Fetchinson wrote: http://www.nightsong.com/phr/crypto/p3.py Thanks a lot, currently I'm having trouble using this code on python 2.6 but probably some small tweaking will fix it. If you keep having issues with this module, maybe you can try this: http://www.freenet.org.nz/ezPyCrypto/ It provides a very easy compact api on top of PyCrypto. So you'll need to install that as well to be able to use this. -irmen -- http://mail.python.org/mailman/listinfo/python-list
Re: lightweight encryption of text file
On 2010-01-08 15:47 PM, Daniel Fetchinson wrote:
I have a plain text file which I would like to protect in a very
simple minded, yet for my purposes sufficient, way. I'd like to
encrypt/convert it into a binary file in such a way that possession of
a password allows anyone to convert it back into the original text
file while not possessing the password one would only see the
following with the standard linux utility 'file':
[fetchin...@fetch ~]$ file encrypted.data
encrypted.data: data
and the effort required to convert the file back to the original text
file without the password would be equivalent to guessing the
password.
I'm fully aware of the security implications of this loose
specification, but for my purposes this would be a good solution.
What would be the simplest way to achieve this using preferably stock
python without 3rd party modules? If a not too complex 3rd party
module made it really simple that would be acceptable too.
Paul Rubin's p3.py algorithm is probably the most straightforward way to
meet
these requirements. It's not a standard crypto algorithm by any means,
but
Paul
knows his stuff and has devised it with these deployment restrictions in
mind.
http://www.nightsong.com/phr/crypto/p3.py
Thanks a lot, currently I'm having trouble using this code on python
2.6 but probably some small tweaking will fix it.
Actually, it also doesn't work with python 2.5 and currently I don't
have access to anything older. array.array raises a
ValueError: string length not a multiple of item size
Does anyone recall a change to array.array?
The full traceback is
Traceback (most recent call last):
File p3.py, line 163, inmodule
_test()
File p3.py, line 143, in _test
c1 = e(plain,key)
File p3.py, line 69, in p3_encrypt
xkey = _expand_key(k_enc, n+4)
File p3.py, line 41, in _expand_key
return array ('L', j)
ValueError: string length not a multiple of item size
Are you on a 64-bit platform? Unfortunately, array's integer typecodes are
platform-specific, but p3.py requires a 32-bit integer and was written on a
32-bit platform. It's reasonably straightforward to fix. Put this bit of
(untested) code at the top of the file and replace occurrences of 'L' with uint32:
# Find the typecode of a 32-bit unsigned integer.
for typecode in 'IL':
if len(array(typecode, [0]).tostring()) == 4:
uint32 = typecode
break
else:
raise RuntimeError(Neither 'I' nor 'L' are unsigned 32-bit integers.)
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth.
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
24 bit signed integer binary conversion help needed
thanks Grant, your sext24 function does the trick (this Python newbie thanks you a lot ...) so i loop on d=wave.readframes(1) call dd=sext24(d) on the frame read , then unpack(dd) the output of sext24 into a 32bit integer (seems to do the trick correctly for signed 24 bit integers ) On 2010-01-08, Robert Somerville rsomerville at sjgeophysics.com wrote: I am trying to read 24bit signed WAV format (little endian) data from a WAV file and convert it to 32 bit little endian integer format ... can anybody please tell me how to do the conversion from 24 bit to 32 bit with a snippet of Python code ??? def sext24(d): if ord(d[2]) 0x80: return d+'\xff' else: return d+'\x00' -- http://mail.python.org/mailman/listinfo/python-list
