Re: Simple text parsing gets difficult when line continues to next line
John Machin wrote:
Jacob Rael wrote:
Hello,
I have a simple script to parse a text file (a visual basic program)
and convert key parts to tcl. Since I am only working on specific
sections and I need it quick, I decided not to learn/try a full blown
parsing module. My simple script works well until it runs into
functions that straddle multiple lines. For example:
Call mass_write(H0, HF, H4, H0, H5, H0, H6, H0, H7, H0,
H8, H0, _
H9, H0, HA, H0, HB, H0, HC, H0, HD, H0, HE,
H0, HF, H0, -1)
I read in each line with:
for line in open(fileName).readlines():
I would line to identify if a line continues (if line.endswith('_'))
and concate with the next line:
line = line + nextLine
How can I get the next line when I am in a for loop using readlines?
Don't do that. I'm rather dubious about approaches that try to grab the
next line on the fly e.g. fp.next(). Here's a function that takes a
list of lines and returns another with all trailing whitespace removed
and the continued lines glued together. It uses a simple state machine
approach.
I agree that mixing the line assembly and parsing is probably a mistake
although using next explicitly is fine as long as your careful with it.
For instance, I would be wary to use the mixed for-loop, next strategy
that some of the previous posts suggested. Here's a different,
generator-based implementation of the same idea that, for better or for
worse is considerably less verbose:
def continue_join_2(linesin):
getline = iter(linesin).next
while True:
buffer = getline().rstrip()
try:
while buffer.endswith('_'):
buffer = buffer[:-1] + getline().rstrip()
except StopIteration:
raise ValueError(last line is continued: %r % line)
yield buffer
-tim
[SNIP]
--
http://mail.python.org/mailman/listinfo/python-list
Re: numpy: frequencies
Filip Wasilewski wrote: robert wrote: I have an integer array with values limited to range(a,b) like: ia=array([1,2,3,3,3,4,...2,0,1]) and want to speedly count the frequencies of the integers into get a density matrix. Is this possible without looping? See numpy.bincount (for integers = 0) if you mean 'without writing looping code in Python' or please specify your question. Question 2: is it possible to compute a moving maximum without python looping ia=array([4,2,1,5,3,2,2,0,1,1]) - mvmax(ia,3) - [4,4,4,5,5,5,3,2,2,1]) I haven't seen a ready solution but this can be easily converted into Pyrex/C looping. I don't know a way to avoid looping entirely, but there are ways that you can loop over the width of the window (in this case 3) rather than over the entire array. Since the window width is generally small compared to the array, this will probably be fast enough. The tricky part is to get the value right at the edges, since what you do there depends on what boundary conditions you apply. The general idea is this: result = ia[n-1:] for i in range(n-1): numpy.maximum(result, ia[i:-n+i], result) This punts on dealing with the ends (and I haven't tested this version), but should give you the idea. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: numpy magic: cast scalar returns auto to python types float int ?
robert wrote: Turning algs for old NumPy modules into numpy code I suffer from this: Upon further processing of returns of numpy calculations, lots of data in an apps object tree will become elementary numpy types. First there is some inefficiency in calculations. And then you get data inflation and questionable dependencies - e.g. with pickle,ZODB,mpi's ... : l=array((1.,0)) l.prod() 0.0 cPickle.dumps(_) cnumpy.core.multiarray\nscalar\np1\n(cnumpy\ndtype\np2\n(S'f8'\nI0\nI1\ntRp3\n(I2\nS''\nNNNI-1\nI-1\ntbS'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00'\ntRp4\n. cPickle.dumps(0.0) 'F0\n.' l=array((1,0)) l.prod() 0 cPickle.dumps(_) cnumpy.core.multiarray\nscalar\np1\n(cnumpy\ndtype\np2\n(S'i4'\nI0\nI1\ntRp3\n(I2\nS''\nNNNI-1\nI-1\ntbS'\\x00\\x00\\x00\\x00'\ntRp4\n. cPickle.dumps(0) 'I0\n.' type(l.prod()) type 'numpy.int32' To avoid this you'd need a type cast in Python code everywhere you get scalars from numpy into a python variable. Error prone task. Or check/re-render your whole object tree. Wouldn't it be much better if numpy would return Python scalars for float64 (maybe even for float32) and int32, int64 ... where possible? (as numarray and Numeric did) I suppose numpy knows internally very quickly how to cast. The short answer is no, it would not be better. There are some trade offs involved here, but overall, always returning numpy scalars is a significant improvement over returning Python scalars some of the time. Which is why numpy does it that way now; it was a conscious choice, it didn't just happen. Please search the archives of numpy-discussion for previous discussions of this and if that is not enlightening enough please ask at on the numpy-discussion list (the address of which just changed and I don't have it handy, but I'm sure you can find it). For your particular issue, you might try tweaking pickle to convert int64 objects to int objects. Assuming of course that you have enough of these to matter, otherwise, I suggest just leaving things alone. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I speed this function up?
Chris wrote:
This is just some dummy code to mimic what's being done in the real
code. The actual code is python which is used as a scripting language in
a third party app. The data structure returned by the app is more or
less like the data list in the code below. The test for ELEMENT is
necessary ... it just evaluates to true every time in this test code. In
the real app perhaps 90% of tests will also be true.
So my question is how can I speed up what's happening inside the
function write_data()? Only allowed to use vanilla python (no psycho or
other libraries outside of a vanilla python install).
Try collecting your output into bigger chunks before writing it out. For
example, take a look at:
def write_data2(out, data):
buffer = []
append = buffer.append
extend = buffer.extend
for i in data:
if i[0] == 'ELEMENT':
append(ELEMENT %06d % i[1])
extend(map(str, i[2]))
append('\n')
out.write(''.join(buffer))
def write_data3(out, data):
buffer = []
append = buffer.append
for i in data:
if i[0] == 'ELEMENT':
append((ELEMENT %06d %s % (i[1],' '.join(map(str,i[2])
out.write('\n'.join(buffer))
Both of these run almost twice as fast as the original below (although
admittedly I didn't check that they were actually right). Using some of
the other suggestions mentioned in this thread may make things better
still. It's possible that some intermediate chunk size might be better
than collecting everything into one string, I dunno.
cStringIO might be helpful here as a buffer instead of using lists, but
I don't have time to try it right now.
-tim
I have a vested interest in showing a colleague that a python app can
yield results in a time comparable to his C-app, which he feels is mch
faster. I'd like to know what I can do within the constraints of the
python language to get the best speed possible. Hope someone can help.
def write_data1(out, data):
for i in data:
if i[0] is 'ELEMENT':
out.write(%s %06d % (i[0], i[1]))
for j in i[2]:
out.write(%d % (j))
out.write(\n)
import timeit
# basic data mimicing data returned from 3rd party app
data = []
for i in range(50):
data.append((ELEMENT, i, (1,2,3,4,5,6)))
# write data out to file
fname = test2.txt
out = open(fname,'w')
start= timeit.time.clock()
write_data2(out, data)
out.close()
print timeit.time.clock()-start
--
http://mail.python.org/mailman/listinfo/python-list
Re: PEP 359: The make Statement
Carl Banks wrote: Mike Orr wrote: I think this PEP is going off the rails. It's primary virtue was that it was a simpler, clearer way to write: class Foo(args): __metaclass__ = some_metaclass #... And it doesn't even do that. What's wrong with class Foo: __metaclass__ = blah? Two lines of code, and the double underscores indicate something special is happening. I think you're missing the point somewhat. The real point isn't to make using metaclasses easier; it's to let the useful semantics of the class statement be used for things that aren't classes. I can see how you might get the impression from the above paragraph, but you'd be wrong. Example: I can define the following metaclass: def PropertyMaker(name,bases,pdict): fget = pdict.get(get,None) fset = pdict.get(set,None) fdel = pdict.get(delete,None) doc = pdict.get(__doc__,None) return property(fget,fset,fdel,doc) Then, I could define a property inside some class definition like this: class some_attribute: __metaclass__ = PropertyMaker def get(self): whatever def set(self,value): whatever But the thing is, if I did that, I'd be lying bastard. I'd be using the class statement and the __metaclass__ property; however, the object I'm creating is not a class (it's a property), and the thing I'm assigning to __metaclass__ is not a metaclass (it's a factory function). With the make statement, I could instead write: make property some_attribute: def get(self): # etc. Then I'm not lying about it, and I'm using a more straightforward syntax. If this proposal were just about metaclasses, I agree that wouldn't be important enough to justify a new statement. Metaclasses aren't too common, and are generally used by experts who don't need the straightforwardness the make statement would provide. But, properties, dicts, and other things could benefit from some of the semantics the class statement, and with the make statement, the average user could take advantage of that without having to worry about all this circumlocative metaclass hackiness. Here you've missed the point of my post. The post itself was not all that clear, I admit, but had you read the subsequent followups, it would have been clear that I wasn't arguing against the utility of the statement (nor was I arguing for it), I was arguing against complicating it for a useless use case. In particular making the namespace associated with make statement into some sort of quasi namespace in order to support the generating of HTML seems foolish for two reasons. First, it doesn't actually work in any but the most trivial of cases. Second, there is an existing syntax (as of 2.5) that can fill the same roll, only it actually works, specifically the 'with' statement. Regards, -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 359: The make Statement
Carl Banks wrote: Tim Hochberg wrote: Carl Banks wrote: Mike Orr wrote: I think this PEP is going off the rails. It's primary virtue was that it was a simpler, clearer way to write: class Foo(args): __metaclass__ = some_metaclass #... And it doesn't even do that. What's wrong with class Foo: __metaclass__ = blah? Two lines of code, and the double underscores indicate something special is happening. I think you're missing the point somewhat. The real point isn't to make using metaclasses easier; it's to let the useful semantics of the class statement be used for things that aren't classes. I can see how you might get the impression from the above paragraph, but you'd be wrong. ??? From the above post, I got the impression that it was Mike Orr that wrote it, not you. If you and he are really the same person, you must admit I would have no reasonable way to get any other impression. :) No really, are you sure I was replying to what you think I was replying to? I totally agree with you about the XML thing; it'd be a terrible misuse of the make statement. But the post I responded to had nothing to do with that. My bad. I had a stack overflow or something when reading the nesting of the post and missed who was replying to what. Sorry, -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: PEP 359: The make Statement
Tim Hochberg wrote:
Steven Bethard wrote:
Steven Bethard wrote:
Tim Hochberg wrote:
Steven Bethard wrote:
Steven Bethard wrote:
Duncan Booth wrote:
make Element html:
make Element body:
make Element p:
text('But this ')
make Element strong:
text('could')
text(' be made to work')
This is nice. I'll have to play around with it a bit to see how
hard it would be to make it work.
Okay, I think it'll work[1]. I'm going to update this section to
something more like:
Open Issues
===
...
Should users of the make statement be able to determine in which dict
object the code is executed? This would allow the make statement to
be used in situations where a normal dict object would not suffice,
e.g. if order and repeated names must be allowed. Allowing this sort
of customization could allow XML to be written like::
I think this PEP is going off the rails. It's primary virtue was
that it was a simpler, clearer way to write:
class Foo(args):
__metaclass__ = some_metaclass
#...
Once it starts calling secret magic methods behind the scenes it's
losing that virture. And for what? Supporting some use cases that
have reasonable solutions already?
That's why it's in the Open Issues section. I expect most of these
open issues to be resolved by rejection. (At least, that's my
preferred resolution.) But since they people have brought them up,
I think they need to be addressed as completely as possible.
But I think you make a good point that this particular case can be
just as easily done using a with-statement (I think). I'll add that
to this part of the PEP (once I'm sure it works).
Hmm... Actually, while the with-statement can probably help out with
the nesting, it doesn't help out with the DRY; you still have to
repeat the element name (once for the call to Element, and once as
the name that Element object is bound to).
I don't think that's correct. I think that with a suitably designed
HtmlDocument object, the following should be possible:
with HtmlDocument(Title) as doc:
with doc.element(body):
doc.text(before first h1)
with doc.element(h1, style=first):
doc.text(first h1)
# I don't understand the point of tail, but you could do that too
doc.text(after first h1)
with doc.element(h1, style=second):
doc.text(second h1)
doc.text(after second h1)
Here's code to do this. It would be probably be better to use elment
tree or some such instead of pushing out the HTML directly, but this
should get the idea across (testing using 2.5a1):
class HtmlTag(object):
def __init__(self, document, type, attribs):
self.document = document
self.type = type
self.attribs = attribs
def __context__(self):
return self
def _format_attribs(self):
if not self.attribs:
return ''
return ' ' + ' '.join('%s=%s' % (k,v) for
k, v in self.attribs.items())
def __enter__(self):
self.document.entities.append('%s%s' % (self.type,
self._format_attribs()))
return self
def __exit__(self, type, value, traceback):
self.document.entities.append('/%s' % self.type)
def add(self):
self.document.entities.append('%s%s/' % (self.type,
self._format_attribs()))
class HtmlDocument(object):
def __init__(self):
self.entities = []
def tag(self, type, **atribs):
return HtmlTag(self, type, atribs)
def text(self, value):
self.entities.append(value)
def tostring(self):
return ''.join(self.entities)
def __context__(self):
return self
def __enter__(self):
self.entities.append('html')
return self
def __exit__(self, type, value, traceback):
if not (type is value is traceback is None):
raise type(value)
self.entities.append('/html')
And here are some simple examples:
with HtmlDocument() as doc:
with doc.tag(body):
doc.text(before first h1)
with doc.tag(h1, style=first):
doc.text(first h1)
doc.text(after first h1)
with doc.tag(h1, style=second):
doc.text(second h1)
doc.text(after second h1)
expected = '''\
html\
body\
before first h1\
h1 style=firstfirst h1/h1\
after first h1\
h1 style=secondsecond h1/h1\
after second h1\
/body\
/html\
'''
print doc.tostring() == expected
some_list = [foo, bar, 'bazz', froodle]
with HtmlDocument() as doc:
with doc.tag(body):
with doc.tag(h1):
doc.text(An ordered list
Re: PEP 359: The make Statement
Steven Bethard wrote:
Steven Bethard wrote:
Duncan Booth wrote:
Steven Bethard wrote:
Should users of the make statement be able to determine in which dict
object the code is executed? The make statement could look for a
``__make_dict__`` attribute and call it to allow things like::
make Element html:
make Element body:
make Element h1:
'''First heading text'''
make Element h1:
'''Second heading text'''
[snip]
There is another effect which should be considered here. If you allow
Element to create an object to be used as the namespace, then as well
as doing special tracking when values are set in the namespace it can
also pre-seed it with names which magically appear in scope within the
make.
e.g.
make Element html:
make Element body:
make Element p:
text('But this ')
make Element strong:
text('could')
text(' be made to work')
This is nice. I'll have to play around with it a bit to see how hard it
would be to make it work.
Okay, I think it'll work[1]. I'm going to update this section to
something more like:
Open Issues
===
...
Should users of the make statement be able to determine in which dict
object the code is executed? This would allow the make statement to
be used in situations where a normal dict object would not suffice,
e.g. if order and repeated names must be allowed. Allowing this sort
of customization could allow XML to be written like::
I think this PEP is going off the rails. It's primary virtue was that it
was a simpler, clearer way to write:
class Foo(args):
__metaclass__ = some_metaclass
#...
Once it starts calling secret magic methods behind the scenes it's
losing that virture. And for what? Supporting some use cases that have
reasonable solutions already?
make Element html:
make Element body:
text('before first h1')
make Element h1:
attrib(style='first')
text('first h1')
tail('after first h1')
make Element h1:
attrib(style='second')
text('second h1')
tail('after second h1')
What's the virtue of this form? Is it the indentation? If so, I suspect
some relatively pretty solution could be manufactured using the 'with'
syntax. Does anyone really write html 'by hand' in this manner anyway? I
think this approach may start looking a lot less appealing when you are
generating the HTML from some other data. Then you have loops inside the
body. Somehow you have to clean up all the leftover loop variables so
they don't end up in your document. However, that would require some
surgery and increased complexity in Element. It all looks like it would
be messy and perhaps useless in real life.
Regards,
-tim
assert etree.ElementTree.tostring(body) == '''\
html\
body\
before first h1\
h1 style=firstfirst h1/h1\
after first h1\
h1 style=secondsecond h1/h1\
after second h1\
/body\
/html\
'''
Assuming that the make statement calls the callable's
``__make_dict__`` to get the dict in which to execute the code, the
following should make the above make statements work::
class Element(object):
class __make_dict__(dict):
def __init__(self, *args, **kwargs):
self._super = super(Element.__make_dict__, self)
self._super.__init__(*args, **kwargs)
self.elements = []
self.text = None
self.tail = None
self.attrib = {}
def __getitem__(self, name):
try:
return self._super.__getitem__(name)
except KeyError:
if name in ['attrib', 'text', 'tail']:
return getattr(self, 'set_%s' % name)
else:
return globals()[name]
def __setitem__(self, name, value):
self._super.__setitem__(name, value)
self.elements.append(value)
def set_attrib(self, **kwargs):
self.attrib = kwargs
def set_text(self, text):
self.text = text
def set_tail(self, text):
self.tail = text
def __new__(cls, name, args, edict):
get_element = etree.ElementTree.Element
result = get_element(name, attrib=edict.attrib)
result.text = edict.text
result.tail = edict.tail
for element in edict.elements:
result.append(element)
return result
[1] Here's the code I used to test it.
def make(callable, name, args, block_string):
...
Re: PEP 359: The make Statement
Steven Bethard wrote:
Steven Bethard wrote:
Tim Hochberg wrote:
Steven Bethard wrote:
Steven Bethard wrote:
Duncan Booth wrote:
make Element html:
make Element body:
make Element p:
text('But this ')
make Element strong:
text('could')
text(' be made to work')
This is nice. I'll have to play around with it a bit to see how
hard it would be to make it work.
Okay, I think it'll work[1]. I'm going to update this section to
something more like:
Open Issues
===
...
Should users of the make statement be able to determine in which dict
object the code is executed? This would allow the make statement to
be used in situations where a normal dict object would not suffice,
e.g. if order and repeated names must be allowed. Allowing this sort
of customization could allow XML to be written like::
I think this PEP is going off the rails. It's primary virtue was that
it was a simpler, clearer way to write:
class Foo(args):
__metaclass__ = some_metaclass
#...
Once it starts calling secret magic methods behind the scenes it's
losing that virture. And for what? Supporting some use cases that have
reasonable solutions already?
That's why it's in the Open Issues section. I expect most of these open
issues to be resolved by rejection. (At least, that's my preferred
resolution.) But since they people have brought them up, I think they
need to be addressed as completely as possible.
But I think you make a good point that this particular case can be just
as easily done using a with-statement (I think). I'll add that to this
part of the PEP (once I'm sure it works).
Hmm... Actually, while the with-statement can probably help out with
the nesting, it doesn't help out with the DRY; you still have to repeat
the element name (once for the call to Element, and once as the name
that Element object is bound to).
I don't think that's correct. I think that with a suitably designed
HtmlDocument object, the following should be possible:
with HtmlDocument(Title) as doc:
with doc.element(body):
doc.text(before first h1)
with doc.element(h1, style=first):
doc.text(first h1)
# I don't understand the point of tail, but you could do that too
doc.text(after first h1)
with doc.element(h1, style=second):
doc.text(second h1)
doc.text(after second h1)
That seems reasonably DRY compliant. Doc would simply stack and unstack
the attributes on the way in and out of the with blocks. This arguably
maps better to the underlying HTML as well.
I'd like to reiterate my point that, as far as I can tell, the make
statement won't actually work for creating HTML in all but the most
trivial of cases. Consider:
with HtmlDocument(Title Here) as doc:
with doc.element(body):
with doc.element(h1):
doc.text(An ordered list)
with doc.element(ol): # Ordered list
for value in some_list:
doc.listitem(value)
If I try to translate this to make syntax, I get into trouble. I end up
with 'value' getting set as an element over and over again. Not at all
what I want! I suppose you could filter your expression so that only
things that subtype some marker class get added, but that's piling
weirdness on top of confusion; and good sign that the whole idea is
better off abandoned.
-tim
--
http://mail.python.org/mailman/listinfo/python-list
Re: unboundlocalerror with cgi module
Kent Johnson wrote:
David Bear wrote:
I'm attempting to use the cgi module with code like this:
import cgi
fo = cgi.FieldStorage()
# form field names are in the form if 'name:part'
keys = fo.keys()
for i in keys:
try:
item,value=i.split(':')
except NameError, UnboundLocalError:
print exception...
item,value=(None,None)
return(item,value)
However, the except block does not seem to catch the exception and an
unboundlocalerror is thrown anyway. What am I missing?
I don't know why you would get an UnboundLocalError from the above code,
but the correct syntax is
except (NameError, UnboundLocalError):
One possibility is that there are no keys. Then the loop finishes
without ever setting item or values. This would give an unbound local
error. I assume that the OP would have noticed that in the traceback,
but perhaps he missed it.
OT, using i for the field namelike that makes my head hurt. Perhaps for
fieldname in fo.keys(): would be in order instead.
Regards,
-tim
Your code is binding the actual exception to the name UnboundLocalError
Kent
--
http://mail.python.org/mailman/listinfo/python-list
Re: unboundlocalerror with cgi module
Tim Hochberg wrote:
Kent Johnson wrote:
David Bear wrote:
I'm attempting to use the cgi module with code like this:
import cgi
fo = cgi.FieldStorage()
# form field names are in the form if 'name:part'
keys = fo.keys()
for i in keys:
try:
item,value=i.split(':')
except NameError, UnboundLocalError:
print exception...
item,value=(None,None)
return(item,value)
However, the except block does not seem to catch the exception and an
unboundlocalerror is thrown anyway. What am I missing?
I don't know why you would get an UnboundLocalError from the above code,
but the correct syntax is
except (NameError, UnboundLocalError):
One possibility is that there are no keys. Then the loop finishes
without ever setting item or values. This would give an unbound local
error. I assume that the OP would have noticed that in the traceback,
but perhaps he missed it.
OT, using i for the field namelike that makes my head hurt. Perhaps for
fieldname in fo.keys(): would be in order instead.
This is bugging me. What's the above supposed to do? Right now, it
iterates over all of the keys and returns item,value for the last key.
That doesn't seem like that's what you want. Is it supposed to return a
sequence of item, value pairs? If that's the case, I think you could
just use:
return [fieldname.split(':') for fieldname in fo]
Or, if you just want the last one, you could just tack a [-1] on the end
of that, although it'll still bomb if there's no keys, so you might want
to check.
The above assumes that FieldStorage faithfully implements the iter part
of the dictionary protocol, something I'm not entirely sure about.
Regards,
-tim
Your code is binding the actual exception to the name UnboundLocalError
Kent
--
http://mail.python.org/mailman/listinfo/python-list
Re: unboundlocalerror with cgi module
Kent Johnson wrote:
Tim Hochberg wrote:
Kent Johnson wrote:
David Bear wrote:
I'm attempting to use the cgi module with code like this:
import cgi
fo = cgi.FieldStorage()
# form field names are in the form if 'name:part'
keys = fo.keys()
for i in keys:
try:
item,value=i.split(':')
except NameError, UnboundLocalError:
print exception...
item,value=(None,None)
return(item,value)
However, the except block does not seem to catch the exception and an
unboundlocalerror is thrown anyway. What am I missing?
I don't know why you would get an UnboundLocalError from the above code,
but the correct syntax is
except (NameError, UnboundLocalError):
One possibility is that there are no keys. Then the loop finishes
without ever setting item or values. This would give an unbound local
error. I assume that the OP would have noticed that in the traceback,
but perhaps he missed it.
I think that would be a NameError for the code shown because item and
value are global variables.
Note that there is a return at the end of the posted code, so I suspect
that this problem actually occurs in a function somewhere and the OP
simplified things in order to post it.
But anyway you raise a good point that
perhaps the reason the exception is not being caught is because it is
raised by code outside the scope of the try.
Also UnboundLocalError is a subclass of NameError so catching NameError
should catch UnboundLocalError as well.
Good point.
-tim
--
http://mail.python.org/mailman/listinfo/python-list
Re: Cheese Shop: some history for the new-comers
[EMAIL PROTECTED] wrote: richard wrote: [snip] Should the Python Cheeseshop have anything in it, though? Having a stocked cheese shop in relation to Python is just silly! Well, it shouldn't have any *cheese*, but that's probably OK for a software repository. -- http://mail.python.org/mailman/listinfo/python-list
Re: import and shared global variables
Michael Brenner wrote: Hi, I'm implementing a plugin-based program, structured like the example below (where m1 in the main module, loading m2 as a plugin). I wanted to use a single global variable (m1.glob in the example) to store some config data that the plugins can access. However, the output shown belown seems to imply that glob is *copied* or recreated during the import in m2. Am I missing something? I thought m1 should be in sys.modules and not be recreated during the import in m2. After browsing c.l.p, it seems that this is probably somehow due to the circular import. However, I do not really see why this should be a problem here. Interestingly, the problem disappears when I put the code in m1 in a real main() function instead of if __name__ etc. Though this seems to solve my problem, I still want to understand what's What happens here is that there does end up being two copies of m1: the one named __main__ and the one imported as m1. If you think about this, there has to be two copies -- otherwise how could sometimes __name__ be __main__ and sometimes not. Anyway, there are several options. The simplest one here is not to modify anything locally from __main__ block. Instead import m1, and modify that copy. That is: glob = [1] if __name__ == __main__: import m1 m1.glob.append(2) print m1.main().1:, m1.glob m2 = __import__(m2) m2.test() print m1.main().2:, glob Regards, -tim happening. Thanks, michael m1.py: -- glob = [1] if __name__ == __main__: glob.append(2) print m1.main().1:, glob m2 = __import__(m2) m2.test() print m1.main().2:, glob -- m2.py: -- def test(): import m1 print m2.test():, m1.glob - Output: m1.main().1: [1, 2] m2.test(): [1] m1.main().2: [1, 2] -- http://mail.python.org/mailman/listinfo/python-list
Re: Cryptographically random numbers
Paul Rubin wrote: Tuvas [EMAIL PROTECTED] writes: I've actually done the tests on this one, it's actually faster to use the += than a list, odd as it may sound. Frederik explained the reason; there's an optimization in Python 2.4 that I'd forgotten about, for that specific case. It's not in earlier versions. It's a bit fragile in 2.4: a = '' for x in something: a += g(x) is fast, but if a is aliased, Python can't do the optimization, so a = '' b = a for x in something: a += g(x) is slow. Is this really true? After the first time through the loop, 'a' won't be aliased any more since strings are immutable. After that the loops should be equivalent. I tried this out to see if I could see a timing difference, in case I was missing something, with Python 2.4.1, the following two snippets timed essentially the same for N up to 2**20 (I didn't try any higher): def concat1(): a = '' for x in ' '*N: a += x return a def concat2(): a = '' b = a for x in ' '*N: a += x return a Regards, -tim Figuring out which case to use relies on CPython's reference counting storage allocator (the interpreter keeps track of how many pointers there are to any given object) and so the optimization may not be feasible at all in other implementations which use different storage allocation strategies (e.g. Lisp-style garbage collection). All in all I think it's best to use a completely different approach (something like StringBuffer) but my effort to start a movement here on clpy a couple months ago didn't get anywhere. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python advocacy in scientific computation
Magnus Lycka wrote: Dennis Lee Bieber wrote: [SNIP] For CPU-bound number-crunching, perhaps... For I/O-bound jobs, the GIL is(should be) released when ever a thread is blocked waiting for I/O to complete. I think CPU-bound number-crunching was the big deal in this case. Somehow, I doubt that the OP uses Matlab for I/O-bound jobs. At least if writing threaded applications becomes less error prone in competing languages, this might well be the weak point of Python in the future. I hope to see some clever solution to this from the Python developers. I don't disagree with this, but it's largely irrelevant to CPU-bound number-crunching using numpy and its bretheren. In that case the bulk of the work is going on in the array extensions, in C, and thus the GIL should be released. Whether it actually is released, I can't say -- never having been blessed/cursed with a multiproccessing box, I haven't looked into it. [SNIP] -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Incorrect Decimal-Float behaviour in comparison tests
Fredrik Lundh wrote:
Cassiano, Marco wrote:
I have difficulties with a weird Python 2.4.2 behaviour in comparing
Decimal to Floats.
For Example :
from decimal import Decimal
a=Decimal('3.7')
b=6.3
if a b :print a,b,'a is greater than b - NOT TRUE'
... else: print a,b,'b is greater than a - CORRECT'
...
3.7 6.3 a is greater than b - NOT TRUE
is this
1.0 0
False
also a bug ?
http://www.python.org/doc/ref/comparisons.html
Most other types compare unequal unless they are the
same object; the choice whether one object is considered
smaller or larger than another one is made arbitrarily but
consistently within one execution of a program.
It's not a bug, but it is a misfeature. In 3.0, I'd love to see
nonsensical comparisons raise TypeError, but still keep the ability to
sort lists of heterogeneous objects. One approach would be keep cmp(x,y)
permissive in its arguments while making __gt__ and friends strict.
Sorting would need to fall back __cmp__, if __lt__ either raised a type
error or returned a non int (e.g., numpy arrays). This could probably be
done by munging PyObject_RichCompareBool, but I haven't looked into it
in any detail.
My $0.02
-tim
[SNIP]
--
http://mail.python.org/mailman/listinfo/python-list
Re: number ranges
[Lots of proposals snipped] 90% of my gripes with range disappeared with the addition of enumerate. However, if there's going to be another round of range literal proposals I might as well throw out what seems (to me anyway) like the only halfway obvious choice in the context of Python. 1. a:b:c is equivalent to slice(a,b,c) with the usual rules that missing values are mapped to None: Parentheses would be required where this construct would otherwise be ambiguous. 2. slice(a,b,c) grows an __iter__ method that returns a generator that produces an appropriate sequence of integers. This is easy to define for positive steps, but tricky for negative steps (more on that in a second). Thus: for i in (1::2): print i, = 1 3 5 ... and for i in (:10:3): print i, = 0 3 6 9 The tricky part is deciding what to with negative steps when the start is undefined. It's tempting to use 0 as the start, but I think -1 is actually better if slightly odd. The reason is that I think the following invariant should hold. newlist = [] for i in someslice: try: newlist.append(origlist[i]) except IndexError: break assert(newlist == origlist[slice]) For that to work out, you want: for i in (::-2): print i, = -1 -3 -5 ... etc. I realize that this is quite similar to the rejected PEP 204 syntax, but the addition of generators has reduced many of the problems that that PEP had. I have no great need for range literals, but it seems that if there were to be one, one that fit together with the existing syntax would be better than making up something new. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: number ranges (was Re: Matlab page on scipy wiki)
[EMAIL PROTECTED] wrote: Tim Hochberg wrote: Colin J. Williams wrote: It would be good if the range and slice could be merged in some way, although the extended slice is rather complicated - I don't understand it. The semantics for an extended slicing are as follows. The primary must evaluate to a mapping object, and it is indexed with a key that is constructed from the slice list, as follows. If the slice list contains at least one comma, the key is a tuple containing the conversion of the slice items; otherwise, the conversion of the lone slice item is the key. The conversion of a slice item that is an expression is that expression. The conversion of an ellipsis slice item is the built-in |Ellipsis| object. The conversion of a proper slice is a slice object (see section section 4.2 The standard type hierarchy http://www.network-theory.co.uk/docs/pylang/ref_30.html) whose |start|, |stop| and |step| attributes are the values of the expressions given as lower bound, upper bound and stride, respectively, substituting |None| for missing expressions. [source: http://www.network-theory.co.uk/docs/pylang/ref_60.html] The seems to be a bit of a problem with slicing that needs sorting out. The syntax for a slice list appears to allow multiple slices in a list: extended_slicing ::= primary primaries.html#tok-primary [ slice_list slicings.html#tok-slice_list ] slice_list ::= slice_item slicings.html#tok-slice_item (, slice_item slicings.html#tok-slice_item)* [,] but the current interpreter reports an error: a= range(20) a[slice(3, 9, 2)] [3, 5, 7] a[slice(3, 9, 2), slice(5, 10)] Traceback (most recent call last): File interactive input, line 1, in ? TypeError: list indices must be integers Extended slicing has nothing to do with lists. All that gobbeldy gook is trying to tell you is what the interpreter does with O[1:2:3, 4:5:6] where O is some arbitrary object. What it does is: O[1:2:3, 4:5:6] == O[slice(1,2,3), slice(4,5,6)] == O.__getitem__((slice(1,2,3), slice(4,5,6))) In the case of python lists, __getitem__ doesn't support multiple slice arguments. However, you are free to define your own types which do support multiple slices. That explains it but the current documentation in the reference manual doesn't(I also tried what Colin did when reading that part). And as I mentioned in another post, the current documentation suggests that extended slicing is not supported for sequence type. As it said the primary must be evaluated to mapping object for extended slicing. Though it seems it doesn't matter as all python does now is turn [start:stop:stride] into slice object and pass that to __getitem__, no matter what object it is. I try not to read the reference manual, since it just makes my eyes glaze over, but I believe that the distinction between sequences and mappings referred to only exists for objects defined at the C level. Objects can define getitem two different ways: using sq_item/sq_slice or using mp_subscript. The former functions take integers, the latter takes general PyObjects. So, only the mapping version (mp_*) supports extended slicing. Python class objects always have the mp_subscript defined and it dispatches things to __getitem__. In the past, lists only supported start and stop arguments to slices, so they could get by with using the sq_ functions. Now that they support step arguments as well, it seems that they must define the mapping methods as well, although I admit I haven't checked. You can get an idea of what sq_slice does by looking at the following chunk of code: class ExtendedSlicingDemo(object): def __getitem__(self, key): return key def __getslice__(self, start, stop): return start, stop esd = ExtendedSlicingDemo() print esd[1:2] print esd[1:2:3] print esd[1:2:3, 3:4:5] print esd[1:2:3, ..., 3:4:5] # Note that __getslice__ is deprecated. I think sq_item/sq_slice is still around for efficiency reasons for sequences that don't support extended slicing as well as backwards compatibility / historical reasons. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Changing values of 1st column of a matrix using take(mymatrix, (0, ), axis=1)
Anthony Liu wrote: I am talking about the 'take' method of numarray. See here for more info: http://stsdas.stsci.edu/numarray/Doc/node33.html If I initialize a matrix 'mymatrix' by zeros((3,3), type=Float64), I get a 3*3 matrix of all zeros. Look: from numarray import * mymatrix = zeros((3,3), type=Float64) mymatrix array([[ 0., 0., 0.], [ 0., 0., 0.], [ 0., 0., 0.]]) I want to change the zeros of only the first column to 1's. In other words, I want to get: array([[ 1., 0., 0.], [ 1., 0., 0.], [ 1., 0., 0.]]) The 'take' method is able to take out the first column, but I am not sure if there is a good way of changing all values of the first column to 1's. Of course, I know that I can achieve my goal pretty easily with this: for i in range(3): mymatrix[i,0] = 1.0 I am just wondering if there is an existing numarray method that elegantly does it for me. mymatrix[:,0] = 1.0 __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com -- http://mail.python.org/mailman/listinfo/python-list
Re: number ranges (was Re: Matlab page on scipy wiki)
Colin J. Williams wrote: It would be good if the range and slice could be merged in some way, although the extended slice is rather complicated - I don't understand it. The semantics for an extended slicing are as follows. The primary must evaluate to a mapping object, and it is indexed with a key that is constructed from the slice list, as follows. If the slice list contains at least one comma, the key is a tuple containing the conversion of the slice items; otherwise, the conversion of the lone slice item is the key. The conversion of a slice item that is an expression is that expression. The conversion of an ellipsis slice item is the built-in |Ellipsis| object. The conversion of a proper slice is a slice object (see section section 4.2 The standard type hierarchy http://www.network-theory.co.uk/docs/pylang/ref_30.html) whose |start|, |stop| and |step| attributes are the values of the expressions given as lower bound, upper bound and stride, respectively, substituting |None| for missing expressions. [source: http://www.network-theory.co.uk/docs/pylang/ref_60.html] The seems to be a bit of a problem with slicing that needs sorting out. The syntax for a slice list appears to allow multiple slices in a list: extended_slicing ::= primary primaries.html#tok-primary [ slice_list slicings.html#tok-slice_list ] slice_list ::= slice_item slicings.html#tok-slice_item (, slice_item slicings.html#tok-slice_item)* [,] but the current interpreter reports an error: a= range(20) a[slice(3, 9, 2)] [3, 5, 7] a[slice(3, 9, 2), slice(5, 10)] Traceback (most recent call last): File interactive input, line 1, in ? TypeError: list indices must be integers Extended slicing has nothing to do with lists. All that gobbeldy gook is trying to tell you is what the interpreter does with O[1:2:3, 4:5:6] where O is some arbitrary object. What it does is: O[1:2:3, 4:5:6] == O[slice(1,2,3), slice(4,5,6)] == O.__getitem__((slice(1,2,3), slice(4,5,6))) In the case of python lists, __getitem__ doesn't support multiple slice arguments. However, you are free to define your own types which do support multiple slices. This type of slicing was added to support Numeric originally and as far as I know is still only really used in Numeric and its successors Numarray and Numpy. Since this was originally posted to the numpy list I assume you are familiar with multidimensional indexing of arrays -- that is extended slicing in action. If you really want to learn about it, and for most people it's unnecessary although perhaps entertaining, play with the following class: class ExtendedSlicingDemo(object): def __getitem__(self, key): return key esd = ExtendedSlicingDemo() print esd[1:2:3] print esd[1:2:3, 3:4:5] print esd[1:2:3, ..., 3:4:5] # = slice(1, 2, 3) (slice(1, 2, 3), slice(3, 4, 5)) (slice(1, 2, 3), Ellipsis, slice(3, 4, 5)) -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Shortest prime number program
In the spirit of pointless pessimization and obfuscation I have crushed
something very similar to Alex Martelli's eratosthenes function onto a
single line. It's truly monstrous, but somewhat entertaining [Some
preemptive linebreaks added]:
def short():import itertools as it;D={};g=D.get;return (
q for q in it.count(2) if
D.__setitem__(it.dropwhile(D.__contains__,
xrange(g(q,q)+q,2**30,g(q,q))).next(),g(q,q))
or q not in D and D.pop(q,1))
I'm sure there's a lesson to this somewhere. Something like I need to
find something better to do with my spare time.
-tim
--
http://mail.python.org/mailman/listinfo/python-list
Re: trapping errors in function call syntax
Avi Kak wrote:
Hello:
Suppose I write a function that I want to be called
with ONLY keyword argumnts, how do I raise an
exception should the function get called with
what look like position-specfic arguments?
Any help would be appreciated.
Say you want the signature to be foo(a, b, c) with no positional
arguments allowed. You can do this:
def foo(*args, **kwargs):
a = kwargs.pop('a')
b = kwargs.pop('b')
c = kwargs.pop('c')
if args or kwargs:
raise TypeError(foo takes only keyword arguments: foo(a,b,c))
# ...
Or something similar.
-tim
--
http://mail.python.org/mailman/listinfo/python-list
Re: how do you pronounce 'tuple'?
Erik Max Francis wrote: Roy Smith wrote: A more interesting question is what do you call ()? A none-tuple? Yeah, that's at the point where it _really_ departs from anything remotely mathematical. Don't think I've ever heard the occasion to talk about 0-tuples in any context, though, so I don't think it's something we need to worry about. I'm sure you'd just call them empty tuples or 0-tuples and move on :-). There's only one tuple of length zero, so I just call it Spot. a = () b = () a is b True -- http://mail.python.org/mailman/listinfo/python-list
Re: Shortest prime number program
While not nearly the shortest proposed thus far, I'm fond of: from itertools import count, ifilter def sieve(s=count(2)): while 1:p=s.next();s=ifilter(p.__rmod__,s);yield p It will generate quite a large number of primes before blowing up (at least 50,000 primes, p=611,957) and it's much faster than the other submissions thus far that can generate a more or less arbitrary number of primes. It's still much slower than Alex Martelli's version in the cookbook though. [And yes I know that you can shave off one character by importing ifilter as i, but that's too much ick for too little gain. There may well be other, more fruitful ways to compact it though] -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about idioms for clearing a list
[EMAIL PROTECTED] wrote: Fredrik Lundh wrote: Python now has, what, three built-in mutable collections types: lists, dictionaries, and sets. Dicts and sets both have a clear() method and lists do not. dicts and sets are mappings, and lists are not. mappings don't support slicing. lists do. I am confused. Could you explain this ? I was under the impression said above(mapping don't support slicing), until after I read the language reference. I don't think it is slicing as in the list slicing sense but it does use the term extend slicing. http://www.python.org/doc/2.4.2/ref/slicings.html The semantics for an extended slicing are as follows. The primary must evaluate to a mapping object, and it is indexed with a key that is constructed from the slice list, as follows. If the slice list contains at least one comma, the key is a tuple containing the conversion of the slice items; otherwise, the conversion of the lone slice item is the key. The conversion of a slice item that is an expression is that expression. The conversion of an ellipsis slice item is the built-in Ellipsis object. The conversion of a proper slice is a slice object (see section 3.2) whose start, stop and step attributes are the values of the expressions given as lower bound, upper bound and stride, respectively, substituting None for missing expressions. This is in place to support multidimensional arrays, such as in numpy. If, for instance, you have a 9x9 array A, then A[3:6,3:6] will extract a 3x3 block from the center of it. A[3:6,3:6] is equivalent to A[(slice(3,6,None), slice(3,6,None))] and the resulting tuple gets passed through the mapping interface, but it is not a mapping in any real sense. I don't think there's anything in core Python that uses this and it's not really relevant to this thread. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Scientific Computing with NumPy
mclaugb wrote: Is Scipy the same thing as ScientificPython? I am confused if SciPy is just the new version. they appear to be separate things. Bryan No, Scientific Python is a collection of Python modules that are useful for scientific computing written by Konrad Hinsen. I'm not a user, but you can find information here: http://starship.python.net/~hinsen/ScientificPython/ SciPy is, I believe, a more recent and more ambitious project. In any event it is not directly related to ScientficPython. Information here: http://scipy.org/ Both of these packages are built on top of one of the three extant array extensions for Python. To try to clear up some of the confusion around those, let me summarize the states of these three packages as I understand it: Numeric: This is the origingal array package. Numarray: This was written as a replacement for Numeric. It has improved performance for large arrays. The internals were also simplified and many other improvements were made (arrays were subclassable, numeric signal handling vastly improved, etc). Unfortunately, a side effect of the changes was that small array performance got worse. There was a signifigant chunk of the numeric community for whom this was a deal breaker and as a result there ended up being a split between the Numeric and Numarray communities. Numpy: This is a rewrite of Numeric that incorporates most of the improvements in Numarray. It is designed to bring the two halves of the Python numeric community back together. So far, it seems to have gotten a positive reception. It is currently at 0.9.4 and I expect a stable 1.0 version in relatively short order. If I were starting with Python numerics, this is where I would start, although I've yet to start converting my Numarray based code over. I hope that sheds some light on this. regards, -tim Brendan [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] As of version 0.4.x, Scipy exclusively uses the newer NumPy module instead of the older Numeric module. The confusion is inevitable in this time of transition, but their intent is to standardize on one array package. Brendan -- Brendan Simons mclaugb wrote: This page documents the differences. It seems that NumPy is supported and more recent. http://numeric.scipy.org/ linda.s [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] On 2/6/06, mclaugb [EMAIL PROTECTED] wrote: Has anyone recompiled the Scientific Computing package using NumPy instead of Numeric? I need a least squares algorithm and a Newton Rhaphson algorithm which is contained in Numeric but all the documentation out there says that Numeric is crap and all code should be using NumPy. Thanks, Bryan what is the relationship between Numeric and Numpy? -- http://mail.python.org/mailman/listinfo/python-list
Re: Replace a module variable with a function call
[EMAIL PROTECTED] wrote: I have a module that defines a variable with a constant value and now I need to make that value dynamic, without affecting module clients. In other words, I need to call a function witout using parenthesis. Example: mymod.py-- def value(): return hi client.py-- import mymod print mymod.value Is there a way to do this ? It can be done, but you're not really supposed to be doing this, so it's moderately horrible to do so. If you can manage, I'd avoid doing this, but if you have to, the strategy is to (1) move the contents of your module into a class, (2) use properties to convert what is now a method call into an attribute access and (3) replace the module in sys.modules with an instance of this class. This occasionally get you into trouble, for instance reload no longer works, but it has mostly worked in my limited experience. Since I'm sure that's pretty opaque, here's an example: # file noparens.py import sys class Module(object): def get_value(self): return hi value = property(get_value) sys.modules[__name__] = Module() # interpreter session import noparens noparens.value 'hi' regards, -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: Numarray, numeric, NumPy, scpy_core ??!!
Szabolcs Nagy wrote: Basically all I need is vectors and 3x3 matrices. hmm is numpy really efficient for 3x3 (or 4x4) matrices and vectors? IMHO an optimized matrix4x4 class can be much faster (i'm just guessing here) eg cgtypes is a simple c++ implementation with boost-python wrapper: http://cgkit.sourceforge.net/ If you're operating on a single 3x3 or 4x4 matrix, nothing you do in Python will be efficient, by which I assume you mean not all that far from C speed; the interpreter overhead is simply too high. If you can operate on large blocks of small arrays and vectors at the same time, which one can usually do after consuming sufficient coffee, then yes numpy can be fast if you're careful. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: flatten a level one list
Michael Spencer wrote: Robin Becker schrieb: Is there some smart/fast way to flatten a level one list using the latest iterator/generator idioms. ... David Murmann wrote: Some functions and timings ... Here's one more that's quite fast using Psyco, but only average without it. def flatten6(): n = min(len(xdata), len(ydata)) result = [None] * (2*n) for i in xrange(n): result[2*i] = xdata[i] result[2*i+1] = ydata[i] -tim Here are some more timings of David's functions, and a couple of additional contenders that time faster on my box (I don't have psyco): # From David Murman from itertools import izip xdata = range(1000) ydata = range(1000)[::-1] def flatten1(x, y): return [i for pair in izip(x, y) for i in pair] def flatten2(x, y): return [i for pair in zip(x, y) for i in pair] def flatten3(x, y): res = [] for pair in izip(x, y): for i in pair: res.append(i) return res # New attempts: from itertools import imap def flatten4(x, y): l = [] list(imap(l.extend, izip(x, y))) return l from Tkinter import _flatten def flatten5(x, y): return list(_flatten(zip(x, y))) flatten_funcs = [flatten1, flatten2, flatten3, flatten4, flatten5] def testthem(): flatten1res = flatten_funcs[0](xdata, ydata) for func in flatten_funcs: assert func(xdata, ydata) == flatten1res def timethem(): for func in flatten_funcs: print shell.timefunc(func, xdata, ydata) testthem() timethem() flatten1(...) 704 iterations, 0.71msec per call flatten2(...) 611 iterations, 0.82msec per call flatten3(...) 344 iterations, 1.46msec per call flatten4(...) 1286 iterations, 389.08usec per call flatten5(...) 1219 iterations, 410.24usec per call Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: What is the slickest way to transpose a square list of lists (tuple of tuples)?
Brian van den Broek wrote: Gerard Brunick said unto the world upon 08/01/06 01:27 PM: My way is ugly. These has to be a better way. Thanks, Gerard If you'd posted your way, I might well have seen if I could do it in a nicer fashion. But, since for all I know, my best efforts would result in the approach you already have, I'm unlikely to put the effort in. I suspect I'm not alone. You might do well to show the approach you are unhappy with. Indeed. And it would also make it obvious what you're trying to do as I'm not completely convinced I understand your description. However, if my guess is correct: transposed = zip(*listoflists) is probably hard to beat. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Shane Hathaway wrote:
Andrew Durdin wrote:
On 12/28/05, Shane Hathaway [EMAIL PROTECTED] wrote:
I just found a 125 character solution. It's actually faster and more
readable than the 133 character solution (though it's still obscure.)
Having spent a good deal of time and effort, and not getting below 144
characters, I am now very eager to see how these shorter versions
work, particularly the 6 120-character solutions. I also expect that
someone (I'll certainly give it a try) will produce a yet shorter
version after the contest closes, based on the knowledge from the
winning solutions.
Roberto Alsina fully documented his 120 byte solution here:
http://www.pycs.net/lateral/weblog/2005/12/29.html#P333
My 120 byte solution is equivalent. Many others probably did the same
thing. Python's philosophy of one way to do it seems to be leading us
all down the same road.
However, it still feels like there's room for improvement. I have a 123
byte solution based on a fairly repetitive 21 character string; if I
could compress the representation of the string, it could win.
Apparently there was as someone put out a 119 byte solution. I can't get
to Roberto's code at present (perhaps he's suffering a bandwidth
crisis?), but it seems that my 120 character solution is different. It
sounds like he encoded his as a row at a time and did two loops one for
each character and one for each row. I encoded the data per character
and did three loops one per column, one per character and one per row.
I'm sure that's not clear, so here's the code:
g=''.join;seven_seg=lambda i:g(
g(' _|x|'[ord(~$]m'k{d\x7fo[int(n)])sj]
for n in i for j in(2,1,4))+'\n'for s in(6,0,3))
I've replaced the unprintable characters and added some preemptive
linebreaks so that hopefully this won't get too munged. It's all clear
now, right? Two hints: 6,0,3-row, 2,1,4-column and the 6 and 1 have to
be where they are to exploit the property that the top row only ever has
a center character in it. That way things can be encoded in 7-bits and
fit in a character string that Python likes. The order of the other loop
indices is mostly accidental but not all permutations may work.
I'm off to try to figure out how to do it the other way now, before the
code gets revealed.
-tim
-tim
--
http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Shane Hathaway wrote: André wrote: For the few that might be interested, I will be posting the details of a 117 character long solution to the challenge on my blog http://aroberge.blogspot.com/. Enjoy! You took advantage of prime numbers, enabling you to extract encoded information using a single modulus operation rather than shift and mask. Brilliant strategy. Congratulations. Indeed. Very impressive. And thanks for posting, it was driving me nuts. Now I wonder if this could be adapted to the other, three loop strategy? It's not clear, but there are three characters that are superfulous in the current incarnation of that. Hmmm.. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Shane Hathaway wrote: I just found a 125 character solution. It's actually faster and more readable than the 133 character solution (though it's still obscure.) It depends on Python 2.4. If Python 2.3 compatibility is required for the contest, I have to add 4 characters. I asked, 2.4 is OK. Drat: I had a 128 char solution I was keeping in reserve. Now it's back to the drawing board -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Marius Gedminas wrote:
Jean-Paul Calderone wrote:
On Tue, 27 Dec 2005 14:02:57 -0700, Tim Hochberg [EMAIL PROTECTED] wrote:
Shane Hathaway wrote:
Paul McGuire wrote:
Also, here's another cheat version. (No, 7seg.com does not exist.)
import urllib2
def seven_seg(x):return urllib2.urlopen('http://7seg.com/'+x).read()
And another one from me as well.
class a:
def __eq__(s,o):return 1
seven_seg=lambda i:a()
This is shorter as __eq__=lambda s,o:1.
Or even
class seven_seg(str):__eq__=lambda*a:1
39 characters; passes the test suite. I'm sure it would be
disqualified for cheating, though. :-)
Tricky. That leads to this 30 character gem:
class seven_seg(str):__eq__=id
-tim
But I can't find the first post in this thread... What are you
guys talking about?
http://www.pycontest.net
--
http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Paul McGuire wrote: Shane Hathaway [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] I'm down to 133 characters (counted according to 'wc -c') on a single line. It contains about 11 whitespace characters (depending on what you consider whitespace.) It's way too tricky for my taste, but it's fun to play anyway. Has anyone done better so far? Here's a hint on my strategy: the code contains three large integers. :-) With 1 large integer, I am down to 200 characters on 1 multiline line. I can guess why you have 3 integers vs. 1, but I don't see how that saves you any characters of code. This is interesting. I have a six line solution that is 133 characters long. I use no long integers, which is what's interesting; two distinct solutions with the same length. I played with long integers for a bit, and I can see how one could use a single long integer, but no idea how one would use three. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Shane Hathaway wrote: Tim Hochberg wrote: Paul McGuire wrote: Shane Hathaway [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] I'm down to 133 characters (counted according to 'wc -c') on a single line. It contains about 11 whitespace characters (depending on what you consider whitespace.) It's way too tricky for my taste, but it's fun to play anyway. Has anyone done better so far? Here's a hint on my strategy: the code contains three large integers. :-) With 1 large integer, I am down to 200 characters on 1 multiline line. I can guess why you have 3 integers vs. 1, but I don't see how that saves you any characters of code. This is interesting. I have a six line solution that is 133 characters long. 133 characters according to 'wc -c'? Indeed: C:\...\pycontest_01wc -c seven_seg_8.py 133 seven_seg_8.py I you strip leading and trailing whitespace, it's 122 characters. For comparison, seven_seg_1.py is 816 characters long:) If so, that will surely win style points over my one-line monster. It's pretty readable, at least with the right tabs setting in one's editor, except for the guts of the thing, which consists of 21 characters of marvelously inscrutable goop. But I bet someone's going to do better (without cheating.) I expect so. However, most people, at least those that are talking, seem to be stalling out in the low 130s, so I predict the final winner will be 127-130 characters long. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
py pan wrote: When you guys say 127~150 characters, did you guys mean usinging test_vectors.py in some way? Or there's no import at all? No import at all. The shortest solution reported so far is 131 characters. Getting down to 127 is just a guess as to where the lower bound is likely to be. Note that in principle it's possible to encode the data for how to display a digit in one byte. Thus it's at least theoretically possible to condense all of the information about the string into a string that's 10 bytes long. In practice it turns out to be hard to do that, since a 10 byte string will generally have a representation that is longer than 10 bytes because of the way the escape sequences get printed out. As a result various people seem to be encoding the data in long integers of one sort or another. The data is then extracted using some recipe involving shifts and s. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Shane Hathaway wrote:
Paul McGuire wrote:
Paul McGuire [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
snip
Well *I'm* certainly looking forward to learning some new tricks! My
(non-cheat) version is a comparatively-portly 245, and no alternatives are
popping into my head at the moment!
-- Paul
down to 2 lines, 229 characters
I'm down to 133 characters (counted according to 'wc -c') on a single
line. It contains about 11 whitespace characters (depending on what you
consider whitespace.) It's way too tricky for my taste, but it's fun to
play anyway. Has anyone done better so far? Here's a hint on my
strategy: the code contains three large integers. :-)
I see now how three large integers could be useful, but the best I could
do with that is 136 characters on 1-line. Yesterday that would have been
great, but it's not so hot today.
Also, here's another cheat version. (No, 7seg.com does not exist.)
import urllib2
def seven_seg(x):return urllib2.urlopen('http://7seg.com/'+x).read()
And another one from me as well.
class a:
def __eq__(s,o):return 1
seven_seg=lambda i:a()
-tim
--
http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Jean-Paul Calderone wrote:
On Tue, 27 Dec 2005 14:02:57 -0700, Tim Hochberg [EMAIL PROTECTED] wrote:
Shane Hathaway wrote:
Paul McGuire wrote:
Also, here's another cheat version. (No, 7seg.com does not exist.)
import urllib2
def seven_seg(x):return urllib2.urlopen('http://7seg.com/'+x).read()
And another one from me as well.
class a:
def __eq__(s,o):return 1
seven_seg=lambda i:a()
This is shorter as __eq__=lambda s,o:1.
But I can't find the first post in this thread... What are you
guys talking about?
There's a contest described at http://www.pycontest.net/. People have
been working on two sorts of solutions: 'honest' solutions that actually
do what's described there. The best of these are around 130 characters.
There's also a set of 'cheat' solutions that fool the supplied test
program. I suspect that these will not pass muster when they actually
get submitted, but we'll see I suppose. A couple of people have figured
out how to write these cheating solution extremely compactly (32 bytes).
One of the simpler ones is:
import test;seven_seg=test.test_vectors.get
This will make sense if you look at the kit supplied by the above site.
-tim
--
http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Christian Tismer wrote: [SNIP] And then help me to setup a different contest about content -- chris As usual, I expect that actually having some working code measuring 'Pythonic' length (and I'm sure we could get into all sorts of fun arguments about the exact definition of that) would go a long way towards convincing the next contest thrower who's measuring length to use something a little less distorting than program length. I've been thinking about writing something, but I've been distracted. Perhaps I'll take a stab at it tomorrow and see what I come up with. That being said, the result I've come up with for the contest are pretty cool in a perverse way, at least while there it's spread out over six lines. Once it gets folded up, it become unbearable. And, sadly, it only saves 3 characters. If one was ignoring leading and trailing whitespace the unfolded version would be the much shorter of the two, but ya gotta play with the rules as they is. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: query on python list
[EMAIL PROTECTED] wrote: hi all am searching for a key in a list, am using Found = 0 for item in list: if not key == item: Found = 0 elif key == item: Found =1 Now based on the Found value i ll manipulate the list. but whenever the key and item doesnt match it makes Found = 0, even though i have an item that matches the key, but it overwrites the Found variable to 0, in the next loop when key ! = item, and hence all my calculations are going wrong, If i find a key in my list, some how the variable Found should be 1,till it comes out of the loop. How can i do that. Well, one way would be: Found = False for item in list: if key == item: Found = True break or maybe: for item in list: if key == item: Found = True break else: Found = False but a better way would be: Found = (key in list) -tim thanks in advance yogi -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
[EMAIL PROTECTED] wrote: Currently I'm on 149 characters in urgh one line - 128 without spaces/newlines. (it'd be three characters shorter if it didn't have to end with a \n) It'll be interesting to see what the short 1-line answers look like. I have a hard time seeing how that's done. It'll especially be interesting to see if we're all using similar tricks or if the multiline attemps take a distinctly different tack than the single line attempts. Currently, I'm down to 137 characters now in 6 lines. There's very little left to trim at this point, so I don't know that I'll be able to knock it down any further using my current approach. And since I don't have any other approaches in the wings, I may be about washed up. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Claudio Grondi wrote: Peter Otten wrote: [SNIP] How good is good for import test_vectors seven_seg = test_vectors.test_vectors.get or code using the test suite in general? Peter This started to remind myself about the story of the contest where the shortest program beeing able to output own source was the job to code. The jury had a big trouble to justify why not give the prize to the one who posted an empty file ... I am currently at 39 bytes following the requirements and the principle given above (my module passes the test). Anyone able to beat that? Wow! It'll be interesting to see how to do that. The obvious way gives 53 bytes. Hmmm, I'll have to see what can be done... However they do say We use a very *similar* test suite to either accept or reject pending submissions. (Emphasis mine) So, it's possible, even probable, that the file test.py either won't be available or won't be helpful. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Steven D'Aprano wrote: On Mon, 26 Dec 2005 13:33:42 -0700, Tim Hochberg wrote: Claudio Grondi wrote: I am currently at 39 bytes following the requirements and the principle given above (my module passes the test). Anyone able to beat that? Wow! It'll be interesting to see how to do that. The obvious way gives 53 bytes. Hmmm, I'll have to see what can be done... OBVIOUS??? Your misinterpreting me here. I meant the obvious way to cheat as Claudio describes above. I don't actually think that kind of cheating will work, since it relies on test_vectors.py being present and it assumes that the test vectors used for checking the program are the same as those in test_vectors.py. The documentation suggests that the later at least is not likely. I still haven't figured out how he manages to get his cheating one so short though, the smallest I can get is 43 characters and it looks like this: import test;seven_seg=test.test_vectors.get This works if you run test. It fails if you try to run it standalone since the import order is wrong. The only algorithm obvious to me came to two lines and about 300 characters with all the pruning I could come up with. Changing algorithms made it longer. I am finding it hard enough to credit that half a dozen people are claiming lengths of around 130-180 characters -- 39 or 53 bytes just seems physically impossible to me, especially given that nine of those bytes are the name of the function! In the 130's is definately possible, but I haven't heard of anyone doing better than that. The identity function seven_seg=lambda s:s takes 20 bytes and does nothing. That leaves NINETEEN bytes to implement the functionality. In the immortal words of Bill and Ted: No way dude! You excellent dudes are righteously bodacious! Nah. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Tim Hochberg wrote: [CHOP] import test;seven_seg=test.test_vectors.get This works if you run test. It fails if you try to run it standalone since the import order is wrong. [CHOP] Or maybe not. An earlier version did, but this one seems OK. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Christian Tismer wrote:
Simon Hengel wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
I'm envisioning lots of convoluted one-liners which
are more suitable to a different P-language... :-)
I feel that python is more beautiful and readable, even if you write
short programs.
How about best compromize between shortness and readibility
plus elegance of design?
I would love to choose those criteria for future events. But I'm not
aware of any algorithm that is capable of creating a ranking upon them.
Maybe we can come up with a solution. Any ideas?
Me neither :-)
Maybe a compromize proposal could be like this:
- Squeezing many lines into one using semicola does not help,
the program will be expanded to use one statement per line
- blank lines are allowed and not counted if they are not
needed as part of the code
These two would be easy to acomplish using something like:
def countchars(text):
n = 0
for line in text.split('\n'):
n += len(line.strip())
return n
This would ignore leading and trailing white space as well as blank lines.
Also makes
a=5; b=10
measure as one character longer than
a = 5
b = 10
which can only be good.
- the length of names does not count, unless the code depends on it.
Probably too hard.
Some harmonization procedure might be applied to every solution
before counting lines, in order to avoid spectacular cryptic stuff.
I thought the metric was characters, not lines. At least that's what the
'about' page says. You still get hit by leading whitespace on multiple
line programs though.
-tim
I have no idea whether I'm serious about this.
Having this said, I'm trashing my one-liner :-))
if-it-doesn't-look-like-Python-it-is-not-Python - ly y'rs -- chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Steven D'Aprano wrote: On Sun, 25 Dec 2005 19:14:43 -0500, rbt wrote: Steven D'Aprano wrote: On Sun, 25 Dec 2005 18:05:37 +0100, Simon Hengel wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I'm envisioning lots of convoluted one-liners which are more suitable to a different P-language... :-) I feel that python is more beautiful and readable, even if you write short programs. How about best compromize between shortness and readibility plus elegance of design? I would love to choose those criteria for future events. But I'm not aware of any algorithm that is capable of creating a ranking upon them. What is your algorithm for determining shortest program? Are you counting tokens, lines or characters? Does whitespace count? If whitespace and var names count, these things are going to be ugly :) Yes, but the question is, is two lines and 347 characters ugly enough to win? No. I have 8 lines and 175 chars at present. And, I expect that's gonna get beaten. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Is it necessary to keep the input parameter as 'input'? Reducing that to a single character drops the length of a program by at least 8 characters. Technically it changes the interface of the function, so it's a little bogus, but test.py doesn't check. (Personally I prefer that if be illegal, but if it's legal I'll have to do it). -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: python coding contest
Remi Villatel wrote: André Malo wrote: is two lines and 347 characters ugly enough to win? Nope. 3 lines / 179 chars here :- Yes, it's quite unreadable. I'm in for the second place with 4 lines / 228 chars. (The problem is that I need to find an internet cafe on 28/29th in order to be able to submit) Do your best! I'd really like to see your code. Right now, 179 chars doesn't seem enough for me to write a Hello world. ;-) I'm down to below 160 characters at this point. And 9 lines, so 22 of those characters are leading white space or returns :( I'd really like to get down to 150, but I'm having a hard time figuring where I can do any more compaction. -tim -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray :: multiplying all the elements in 1d array
Mandus wrote: Tue, 20 Dec 2005 19:32:13 +0530 skrev Suresh Jeevanandam: Hi all, Lets say I have an array: from numarray import * a = array([ 6, 7, 8, 9, 10, 11, 12]) I want to multiply out all the elements and get the result. r = 1.0 for i in a: r = r*i Is there any faster, efficient way of doing this. You can use multiply.reduce(a) (multiply is a function imported from numarray). With regular python you can also do: from operator import mul reduce(mul,a) This work even when 'a' is a plain python list. Actually, they'll both work with a plain python list. Multiply.reduce is much faster (see below) when operating on arrays, but somewhat slower when operating on lists. I'd guess from the overhead of converting the list to an array. -tim a = na.arange(10) l = range(10) t0 = time.clock(); na.multiply.reduce(a); print time.clock() - t0 0 0.00388960049392 t0 = time.clock(); na.multiply.reduce(l); print time.clock() - t0 0 0.042208716471 t0 = time.clock(); reduce(mul, a); print time.clock() - t0 0 0.0943455103931 t0 = time.clock(); reduce(mul, l); print time.clock() - t0 0 0.0146099574108 -- http://mail.python.org/mailman/listinfo/python-list
Re: Suggesting a new feature - Inverse Generators
Jordan Rastrick wrote: [CHOP] Behold: # An Acceptor/Generator!!! def combineIntoRecords(): optionalline = None # We may not get given a value for this line accept firstline accept secondline if condition(secondline): accept optionalline accept lastline yield createRecord(firstline, secondline, optionalline, lastline) That's a nice example, but why not: def combineIntoRecords(iterable): iterator = iter(iterable) optionalline = None firstline = iterator.next() secondline = iterator.next() if condition(secondline): optionalline = iterator.next() lastline = iterator.next() yield createRecord(firstline, secondline, optionalline, lastline) ??? -tim def getData(lines): return list(combineIntoRecords(filterJunk(lines))) So, alas, my beloved for loop was not saved after all. Fortunately, the dreaded While loop has been vanquished. In fact all loopiness has disappeared. Along with any other semblance of a main method. I think I like the finished result. I'll leave if for you to decide if you like it too. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic search of list of dictionaries
Bulba! wrote:
Hello everyone,
I'm reading the rows from a CSV file. csv.DictReader puts
those rows into dictionaries.
The actual files contain old and new translations of software
strings. The dictionary containing the row data looks like this:
o={'TermID':'4', 'English':'System Administration',
'Polish':'Zarzadzanie systemem'}
I put those dictionaries into the list:
oldl=[x for x in orig] # where orig=csv.DictReader(ofile ...
..and then search for matching source terms in two loops:
for o in oldl:
for n in newl:
if n['English'] == o['English']:
...
Now, this works. However, not only this is very un-Pythonic, but also
very inefficient: the complexity is O(n**2), so it scales up very
badly.
What I want to know is if there is some elegant and efficient
way of doing this, i.e. finding all the dictionaries dx_1 ... dx_n,
contained in a list (or a dictionary) dy, where dx_i contains
a specific value. Or possibly just the first dx_1 dictionary.
Sure, just do a little preprocessing. Something like (untested):
def make_map(l):
# This assumes that each English key is unique in a given l
# if it's not you'll have to use a list of o instead of o itself.
map = {}
for d in l:
if 'English' in d:
key = d['English']
map[key] = d
old_map = make_map(oldl)
new_map = make_map(newl)
for engphrase in old_map:
if engphrase in new_map:
o = old_map[engphrase]
n = new_map[engphrase]
if n['Polish'] == o['Polish']:
status=''
else:
status='CHANGED'
# process
I've assumed that the English key is unique in both the old and new
lists. If it's not this will need some adjustment. However, your
original algorithm is going to behave weirdly in that case anyway
(spitting out multiple lines with the same id, but potentially different
new terms and update status).
Hope that's useful.
-tim
I HAVE to search for values corresponding to key 'English', since
there are big gaps in both files (i.e. there's a lot of rows
in the old file that do not correspond to the rows in the new
file and vice versa). I don't want to do ugly things like converting
dictionary to a string so I could use string.find() method.
Obviously it does not have to be implemented this way. If
data structures here could be designed in a proper
(Pythonesque ;-) way, great.
I do realize that this resembles doing some operation on
matrixes. But I have never tried doing smth like this in
Python.
#-- Code follows -
import sys
import csv
class excelpoldialect(csv.Dialect):
delimiter=';'
doublequote=True
lineterminator='\r\n'
quotechar=''
quoting=0
skipinitialspace=False
epdialect=excelpoldialect()
csv.register_dialect('excelpol',epdialect)
try:
ofile=open(sys.argv[1],'rb')
except IOError:
print Old file %s could not be opened % (sys.argv[1])
sys.exit(1)
try:
tfile=open(sys.argv[2],'rb')
except IOError:
print New file %s could not be opened % (sys.argv[2])
sys.exit(1)
titles=csv.reader(ofile, dialect='excelpol').next()
orig=csv.DictReader(ofile, titles, dialect='excelpol')
transl=csv.DictReader(tfile, titles, dialect='excelpol')
cfile=open('cmpfile.csv','wb')
titles.append('New')
titles.append('RowChanged')
cm=csv.DictWriter(cfile,titles, dialect='excelpol')
cm.writerow(dict(zip(titles,titles)))
print titles
print -
oldl=[x for x in orig]
newl=[x for x in transl]
all=[]
for o in oldl:
for n in newl:
if n['English'] == o['English']:
if n['Polish'] == o['Polish']:
status=''
else:
status='CHANGED'
combined={'TermID': o['TermID'], 'English': o['English'],
'Polish': o['Polish'], 'New': n['Polish'], 'RowChanged': status}
cm.writerow(combined)
all.append(combined)
# duplicates
dfile=open('dupes.csv','wb')
dupes=csv.DictWriter(dfile,titles,dialect='excelpol')
dupes.writerow(dict(zip(titles,titles)))
for i in xrange(0,len(all)-2):
for j in xrange(i+1, len(all)-1):
if (all[i]['English']==all[j]['English']) and
all[i]['RowChanged']=='CHANGED':
dupes.writerow(all[i])
dupes.writerow(all[j])
cfile.close()
ofile.close()
tfile.close()
dfile.close()
--
Real world is perfectly indifferent to lies that
are the foundation of leftist thinking.
--
http://mail.python.org/mailman/listinfo/python-list
