Re: newbie/ merging lists of lists with items in common
Paddy: _ = [d[x0].append(x1) for x0,x1 in data] I think that you probably want: for text, num in data: d[text].append(num) ardief: thanks to everyone for the help, and the speed of it! It's really useful and I will spend some time working on understanding the code you posted. I'd be so lost without this group... If you have Python 2.5, and if you don't have particular requirements, I think the best solution is Paddy's. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: pil, histogram, mask
Daniel Nogradi I don't need the histogram really, only the mean color value, but as far as I can see the 'mean' attribute only applies to an image and a mask can not be specified. You can slice parts of the image, and then use the ImageStat.Stat(im).mean On it. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Inconsistent list/pointer problem
Doug Stell: The standard module copy has deepcopy, it's slow but it may be a simple solution to your problem. A better solution is to look where data is changed and fix that. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: A* search implementation in Python
Reid Priedhorsky: I'm looking for an open-source Python implementation of A* search for use in a mapping application. You can try this one: http://aima.cs.berkeley.edu/python/search.html Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Overloading the tilde operator?
Peter Otten: No, you will get a syntax error before python even look up the names: There are some tricks that allow the use of undefined symbols in Python too, but they are probably just toys. I have recently posted a recipe in the cookbook for that. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
'First class' Relationships
Currently reading an article, First Class Relationships in an Object- oriented Language, by Gavin Bierman and Alisdair Wren: http://homepages.inf.ed.ac.uk/wadler/fool/program/final/4/4_Paper.pdf Found in the Lambda the Ultimate blog: http://lambda-the-ultimate.org/node/2013 Maybe it can be done in Python too, with a metaclass Relationship. The relation can probably be added dynamically to an object/class. Some typecheek can be added too, if necessary. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Executing Javascript, then reading value
Jean-Paul Calderone: You might look into the stand-alone Spidermonkey runtime. However, it lacks the DOM APIs, so it may not be able to run the JavaScript you are interested in running. There are a couple other JavaScript runtimes available, at least. This may be okay too: http://www.digitalmars.com/dscript/ Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Fixed length lists from .split()?
Duncan Booth: def nsplit(s, sep, n): return (s.split(sep) + []*n)[:n] Another version, longer: from itertools import repeat def nsplit(text, sep, n): nsplit(bcsn; 101; 1456, ;, 3) ['bcsn', ' 101', ' 1456'] nsplit(bcsn; 101, ;, 3) ['bcsn', ' 101', ''] nsplit(bcsn, ;, 3) ['bcsn', '', ''] nsplit(, ., 4) ['', '', '', ''] nsplit(ab.ac.ad.ae, ., 2) ['ab', 'ac', 'ad', 'ae'] result = text.split(sep) nparts = len(result) result.extend(repeat(, n-nparts)) return result if __name__ == __main__: import doctest doctest.testmod() Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: A note on heapq module
bearophile: I don't like your solution, this class was already slow enough. Don't use unbound methods with this class :-) Sorry for raising this discussion after so much time. Another possibile solution is to use the normal methods for the normal case, and replace them only if key is present (this reduces problems with unbound methods but you may use the wrong method). Example: def __init__(self, sequence=None, key=None, inplace=False): ... if key is None: ... else: ... self.pop = self.__pop_key ... def pop(self): return heappop(self._heap) def __pop_key(self): return heappop(self._heap)[2] Someone else has done something similar: http://groups.google.com/group/comp.lang.python/browse_thread/thread/e61ba58c9722bf51/ http://sourceforge.net/tracker/index.php?func=detailaid=1162363group_id=5470atid=305470 Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
dict.keys() ?
The PEP 3100: http://www.python.org/dev/peps/pep-3100/ says: Return iterators instead of lists where appropriate for atomic type methods (e.g. dict.keys(), dict.values(), dict.items(), etc.); iter* methods will be removed. Better: make keys(), etc. return views ala Java collections??? ... To be removed: dict.setdefault()? [15] dict.has_key() method [done] I may be sleepy now, but maybe the keys() method too can be removed; otherwise the following two become really the same: print list(adict) for k in adict: ... print list(adict.keys()) for k in adict.keys(): ... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Fast Imaging for Webserver
Paul McGuire: before you start replacing PIL, or optimizing CherryPy, or other possible performance-improving efforts, you should profile the within-request processing, find the bottleneck, and take care of that first. Good advice. Among the tests, the OP can also try to change the antialiasing type of PIL, going to the nearest neighbor, and doing some transfer timings again. It may help find where the bottleneck is. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: How can i do this in Python?
Gabriel Genellina: import operator,fileinput mapper = map(operator.itemgetter, [0,1,20,21,2,10,12,14,11,4,5,6]) for line in fileinput.input(): fields = line.split() print '\t'.join(m(fields) for m in mapper) I'm just curious, what's the advantage of using itemgetter there compared to something simpler like this (untested!)? import fileinput positions = [0,1,20,21,2,10,12,14,11,4,5,6] for line in fileinput.input(): fields = line.split() print '\t'.join(fields[m] for m in positions) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: str and __setitem__
Peter Otten: class mutable_str(object):... def __init__(self, value): ... self._value = value ... def __setitem__(self, index, value): ... self._value = self._value[:index] + value + self._value[index+1:] ... def __str__(self): ... return self._value ... a = mutable_str(123) a[1] = x print a For this purpose an array may be better, if you have to change it often and print is only once in a while: from array import array a = array(c, 123) a[1] = x print a The OP can also use a class, if some other methods are needed: from array import array class mutable_str(object): def __init__(self, value): self._value = array(c, value) def __setitem__(self, index, value): self._value[index] = value def __str__(self): return self._value.tostring() # this requires time a = mutable_str(123) a[1] = x print a Probably array.array can be subclassed too... bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Static variables
Bruno Desthuilliers: And this let you share state between functions: def make_counter(start_at=0, step=1): count = [start_at] def inc(): count[0] += step return count[0] def reset(): count[0] = [start_at] return count[0] def peek(): return count[0] return inc, reset, peek foo, bar, baaz = make_counter(42, -1) print baaz() for x in range(5): print foo() print bar() print baaz() An interesting solution, I have never created such grouped clorures. I don't know if this solution is better than a class with some class attributes plus some class methods... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Best way to document Python code...
Boris Ozegovic: Does Python has API just like in Java, for example http://java.sun.com/j2se/1.5.0/docs/api/allclasses-noframe.html ctrl-f and than click on class you are searching for, and finally you get clean list of all fields and methods. Where can I find similar in Python, for example, if I would like to see which methods list/dictionary has. You can do that from the shell, with help(name) or dir(name), where name can be a class, object, most things. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Symbols again
Modifying another recipe, I have defined some symbols again: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/500273 I don't know if this may be useful for something real. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Py 2.5 on Language Shootout
[EMAIL PROTECTED]: In reality the choice would be C++ because of OO and STL. I have seen that when Python+Psyco are too much slow, D language is a good sweet half point between Python and C++ :-) Fast as C++ and with a simpler syntax and semantics (Pyrex isn't bad, but D gives high-level things at higher speed). And in the future ShedSkin and RPython may become options too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Match 2 words in a line of file
Rickard Lindberg, yesterday I was sleepy and my solution was wrong. 2) If you have a line something like this: foobar hello then 'foo' in line will return true, even though foo is not a word (it is part of a word). Right. Now I think the best solution is to use __contains__ (in) to quickly find the lines that surely contains both substrings, then on such possibly rare cases you can use a correctly done RE. If the words are uncommon enough, such solution may be fast and reliable. Using raw tests followed by slow and reliable ones on the rare positive results of the first test is a solution commonly used in Computer Science, that often is both fast and reliable. (It breaks when the first test is passed too much often, or when it has some false negatives). Probably there are even faster solutions, scanning the whole text at once instead of inside its lines, but the code becomes too much hairy and probably it's not worth it. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterator length
Steven D'Aprano: s = aaabaabb from itertools import groupby print [(h,leniter(g)) for h,g in groupby(s)] s isn't an iterator. It's a sequence, a string, and an iterable, but not an iterator. If you look better you can see that I use the leniter() on g, not on s. g is the iterator I need to compute the len of. I hope you know what sequences and strings are :-) Well, I know little still about the C implementation of CPython iterators :-) But I agree with the successive things you say, iterators may be very general things, and there are too many drawbacks/dangers, so it's better to keep leniter() as a function separated from len(), with specialized use. Bye and thank you, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterator length
Steven D'Aprano: since g is not an arbitrary iterator, one can easily do this: print [(h,len(list(g))) for h,g in groupby(s)] No need for a special function. If you look at my first post you can see that I have shown that solution too, but it creates a list that may be long, that may use a lot of of memory, and then throws it away each time. I think that's a bad solution. It goes against the phylosophy of iterators too, they are things created to avoid managing true lists of items too. If you, the iterator creator, know enough about the iterator to be sure it has a predictable length, you know how to calculate it. I don't agree, sometimes I know I have a finite iterator, but I may ignore how many elements it gives (and sometimes they may be a lot). See the simple example with the groupby. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Py 2.5 on Language Shootout
The The Computer Language Shootout has just published results for Python 2.5 and Psyco 1.5.2. Comparing the old (Python 2.4) Gentoo Pentium 4 results (now not visible anymore) with the new results, I have seen that all the tests with Python 2.5 are faster than the ones with Python 2.4 (some results can't be compared because N is changed): Gentoo Pentium 4, Python 2.4.3 measurements: Program Logs CPU Time Memory KB GZip N binary-trees 99.26 15,816402 16 chameneos Timout 5.000.000 cheap-concurrency 23,13 5.252160 15.000 fannkuch 66,38 2.200395 10 fasta 81,62 9.884861 2.500.000 k-nucleotide 15,52 15.580459 250.000 mandelbrot363,86 2.412472 3.000 n-bodyTimout 20.000.000 nsieve 9,79 34.416269 9 nsieve-bits 164,72 42.412320 11 partial-sums 38,64 2.300410 2.500.000 pidigits9,22 2.388391 2.500 recursive 701,64 14.360344 11 regex-dna 6,21 24.160326 500.000 reverse-complement 2,7 46.032272 2.500.000 spectral-norm 696,76 2.456266 2.500 startup 6,38 29 200 sum-file8,08 2.364 61 8.000 Regarding Psyco, two tests only are worse (the sourcecode, CPU and SO are the same): Old (Python 2.4.3, older Psyco): nsieve 4.2222,680 211 9 reverse-complement 1.6649,336 330 2,500,000 New (Python 2.5, Psyco 1.5.2): nsieve 4.2622,904 211 9 reverse-complement 1.7552,056 330 2,500,000 Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: A note on heapq module
Steven Bethard: Antoon Pardon: For me, your class has the same drawback as the heappush, heappop procedurers: no way to specify a comparision function. Agreed. I'd love to see something like ``Heap(key=my_key_func)``. It can be done, but the code becomes more complex and hairy: http://rafb.net/p/iCCmDz16.html (If someone spots a problem please tell me, thank you.) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: A note on heapq module
Neil Cerutti: One more idea, cribbed from the linked list thread elsewhere: it might be nice if your Heap could optionally use an underlying collections.deque instead of a list. I don't know how excellent Python's deque is, but it's possible a deque would provide a faster heap than a contiguous array. C++'s std::deque is the default implementation of C++'s std::priority_queue for that reason (unless I'm confused again). If you have some minutes you can do few speed tests and show us the code and the timing results... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: A note on heapq module
Steven Bethard wrote: The current code fails when using unbound methods however:: I don't like your solution, this class was already slow enough. Don't use unbound methods with this class :-) Maybe there's a (better) solution to your problem: to make Heap a function (or classmethod) that return sone of two possibile objects created by one of two different classes that have different methods... Beside that, I think __eq__ method needs more tests, because comparing a Heap with key against another Heap without key may give some problems... I'll think about such problems/things. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Iterator length
Often I need to tell the len of an iterator, this is a stupid example: l = (i for i in xrange(100) if i1) len isn't able to tell it: len(l) Traceback (most recent call last): File stdin, line 1, in module TypeError: object of type 'generator' has no len() This is a bad solution, it may need too much memory, etc: len(list(l)) This is a simple solution in a modern Python: sum(1 for _ in l) 50 This is a faster solution (and Psyco helps even more): def leniter(iterator): leniter(iterator): return the length of an iterator, consuming it. if hasattr(iterator, __len__): return len(iterator) nelements = 0 for _ in iterator: nelements += 1 return nelements Is it a good idea to extend the functionalities of the built-in len function to cover such situation too? Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterator length
George Sakkis: Is this a rhetorical question ? If not, try this: It wasn't a rhetorical question. x = (i for i in xrange(100) if i1) if leniter(x): print x.next() What's your point? Maybe you mean that it consumes the given iterator? I am aware of that, it's written in the function docstring too. But sometimes you don't need the elements of a given iterator, you just need to know how many elements it has. A very simple example: s = aaabaabb from itertools import groupby print [(h,leniter(g)) for h,g in groupby(s)] Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Match 2 words in a line of file
MrJean1 wrote: def lines_with_words(file, word1, word2): Print all lines in file that have both words in it. for line in file: words = line.split() if word1 in words and word2 in words: print line This sounds better, it's probably faster than the RE version, Python 2.5 has a really fast str.__contains__ method, done by effbot: def lines_with_words(file, word1, word2): Print all lines in file that have both words in it. (word1 may be the less frequent word of the two). for line in file: if word1 in line and word2 in line: print line Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I create a linked list in Python?
Marc 'BlackJack' Rintsch: Python has a list type without the s, it's a real list. Don't confuse the *ADT list* with *linked lists* which are just one implementation of the ADT list. Right, I was mostly talking about (single/double) linked lists :-) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
A note on heapq module
In few minutes I have just written this quite raw class, it lacks doctring (the same of the functions of the heapq module), it may contain bugs still, I haven't tested it much. It's just a simple wrapper around some of the functions of the heapq module (nsmallest and nlargest are missing). Usually I use classes only when I need then, I like functional programming enough, and this class seems to just slow down the functions of the heapq module. But I think an improved class similar to this one may be useful (added, replacing isn't necessary) into the heapq module, because it can avoid cetain errors: I know what Heaps are and how they work, but some time ago I have written a bug into small program that uses the functions of the heapq module, because I have added an item to the head of the heap using a normal list method, breaking the heap invariant. With a simple class like this one all the methods of your object keep the heap invariant, avoiding that kind of bugs. (If you are interested I can improve this class some, it't not difficult.) from heapq import heapify, heappush, heappop, heapreplace class Heap(object): def __init__(self, init=None): if init is None: self.h = [] elif isinstance(init, Heap): self.h = init.h[:] else: self.h = list(init) heapify(self.h) def smallest(self): return self.h[0] def sort(self, cmp=None, key=None): self.h.sort(cmp=cmp, key=key) def heappush(self, item): heappush(self.h, item) def heappop(self): return heappop(self.h) def heapreplace(self, item): return heapreplace(self.h, item) def clear(self): del self.h[:] def __contains__(self, item): return item in self.h def __hash__(self): raise TypeError(Heap objects are unhashable.) def __iter__(self): return self.h.__iter__() def __eq__(self, other): return isinstance(other, Heap) and self.h == other.h def __ne__(self, other): return not isinstance(other, Heap) or self.h != other.h def __len__(self): return len(self.h) def __nonzero__(self): return bool(self.h) def __str__(self): return str(self.h) def __repr__(self): return Heap(%s) % self.h if self.h else Heap() Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: A note on heapq module
I think the sort has to be simplified, otherwise it can't keep the heap invariant... def sort(self): self.h.sort() Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: A note on heapq module
Scott David Daniels: I'd suggest some changes. It is nice to have Heaps with equal contents equal no matter what order the inserts have been done. Consider how you want Heap([1, 2, 3]) and Heap([3, 1, 2]) to behave. Similarly, it is nice to have str and repr produce canonical representations (I would skip the __str__ code, myself, though). Also, subclasses should get their identities tweaked as so: My version was a *raw* one, just an idea, this is a bit better: http://rafb.net/p/nLPPjo35.html I like your changes. In the beginning I didn't want to put __eq__ __ne__ methods at all, because they are too much slow, but being them O(n ln n) I think your solution is acceptable. Some methods may have a name different from the heap functions: def smallest(self): def push(self, item): def pop(self): def replace(self, item): Two things left I can see: I think the __init__ may have a boolean inplace parameter to avoid copying the given list, so this class is about as fast as the original heapify function (but maybe such thing is too much dirty for a Python stdlib): def __init__(self, sequence=None, inplace=False): if sequence is None: self._heap = [] elif isinstance(sequence, self.__class__): self._heap = sequence._heap[:] else: if inplace and isinstance(sequence, list): self._heap = sequence else: self._heap = list(sequence) heapify(self._heap) The second thing, I don't like much the __iter__ because it yields unsorted items: def __iter__(self): return self._heap.__iter__() Is this good? I think this can be accepted. Thank you, bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: How can I create a linked list in Python?
Gary Herron: If you really want a list (as Python defines a list - with all the methods) then you should use Python's lists. They are quite efficient and convenient: In CPython they are dynamic arrays, they aren't lists. Try adding elements at the beginning of a list compared to adding elements at the beginning or in the middle of a python list and you will see the efficiency differences. The concept of a linked list if a figment of languages with pointer data types. You may manage a list with a language without pointers, like *basic* (for students) Scheme. Creating a List data type for Python is possible too, without true pointer like ones found in Pascal. However, you're much better off if you can use Python's list data structure rather than try to emulate an outdated concept in a modern language. Simple lists today aren't much efficient because their pointers break cache locality, they may cause cache trashing, so they may be slower than smarter and more localized structures, like short double linked arrays, etc. But I think lists aren't outdated, they are a mathematical concept too, and they are quite used still. If you want to learn some Computer Science, it's positive to know what linked lists are. If you want to implement algorithms that must process LOT of things, you may find pointers and lists quite necessary today too, see for example: http://citeseer.ist.psu.edu/knuth00dancing.html http://en.wikipedia.org/wiki/Dancing_Links Python and its data structures aren't the right solutions for all purposes. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Class list of a module
Gabriel Genellina: import inspect def getClassList(aModule): return [cls for cls in vars(aModule).itervalues() if inspect.isclass(cls)] This is short enough too: from inspect import isclass getclasses = lambda module: filter(isclass, vars(module).itervalues()) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: help with recursion on GP project
First possible raw solution: from operator import add, sub, mul, div, neg def evaluate(expr): if isinstance(expr, list): fun, ops = expr[0], expr[1:] return fun(*map(evaluate, ops)) else: return expr example = [add, [add, [sub, 5, 4], [mul, 3, 2]], [neg, 5]] print evaluate(example) But it's rather slow... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: AlphaBeta Search
Chris wrote: I know this probably seems trivial, but I can't seem to find the bug in my alphabeta search implementation. This is another alphabeta implementation (all the nicest algorithms are inside this AIMA codebase): http://aima.cs.berkeley.edu/python/games.html Later when you program works you can find this helpful to improve your algorithm: http://citeseer.ist.psu.edu/11954.html Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Comparing a matrix (list[][]) ?
Roberto Bonvallet: What output are you expecting from your example matrix? If you are expecting it to be 5 (the smallest positive element), using min is the way to do it: matrix = [[9, 8, 12, 15], ... [0, 11, 15, 18], ... [0, 0, 10, 13], ... [0, 0, 0, 5]] min(min(x for x in row if x 0) for row in matrix) 5 This looks a bit like homework... To find the minimal number 0 of the matrix this seems better: mat = [[9, 8, 12, 15], [0, 11, 15, 18], [0, 0, 10, 13], [0, 0, 0, 5]] print min(el for row in mat for el in row if el 0) Note that this raises TypeError is the matrix doesn't have one or more numbers 0 If the OP needs the position he/she can do something like: nozeromin = 1e300 # or something similar pos = None, None for nrow, row in enumerate(mat): for ncol, el in enumerate(row): if el 0 and el nozeromin: nozeromin = el pos = nrow, ncol print nozeromin, pos Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Type casting a base class to a derived one?
Frederic Rentsch: If I derive a class from another one because I need a few extra features, is there a way to promote the base class to the derived one without having to make copies of all attributes? class Derived (Base): def __init__ (self, base_object): # ( copy all attributes ) ... This looks expensive. No need to copy attributes: class Base(object): def __init__ (self, x): self.x = x class Derived(Base): def __init__ (self, x, y): Base.__init__(self, x) self.y = y d = Derived(1, 2) print d.x, d.y Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: what is the idiom for copy lots of params into self?
Emin: This saves a lot of code and makes it easier to see what is going on, but it seems like there should be a better idiom for this task. Any suggestions? I know two ways of doing it, one way requires very light code into the __init__ and it uses a decorator that takes the parameters and updates self, but I have seen it's not much reliable in the implementation I have. The other way requires a bit more code into the method, but the function code is quite short, easy to understand, and it seems reliable enough (code from web.py, a bit modified): def selfassign(self, locals): for key, value in locals.iteritems(): if key != 'self': setattr(self, key, value) Used on the first line as: def __init__(self, foo, bar, baz=1): selfassign(self, locals()) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: recursive function
First possible solution:
def rloop(seqin, comb):
# xcross product recipe 302478 by David Klaffenbach
if seqin:
for item in seqin[0]:
newcomb = comb + [item]
for item in rloop(seqin[1:], newcomb):
yield item
else:
yield comb
data = {1: [(3, 4), (5, 8)],
2: [(5, 4), (21, 3), (19, 2)],
3: [(16, 1), (0, 2), (1, 2), (3, 4)]}
print [sol for sol in rloop(data.values(), []) if 24==sum(el[0] for el
in sol)]
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why less emphasis on private data?
Paul Rubin: Python certainly makes you spend more of your attention worrying about possible attribute name collisions between classes and their superclasses. And Python's name mangling scheme is leaky and bug-prone if you ever re-use class names. Trouble with this is you can have two classes with the same name, perhaps because they were defined in different modules, and then the name mangling fails to tell them apart. Without changing Python syntax at all I think this situation may be improved. Instead of Python applying name mangling to names with __ before them, it can manage them as private, a higher level kind of management. And then if it's useful a new built-in function may be invented to access such private attributes anyway. I think this may solve your problem. (This is for Py3.0). Maybe a metaclass can be invented to simulate such behavior to test and try it before modifying the language itself. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: I want to learn
[EMAIL PROTECTED]: I am looking for a python guru who has instant messenger or gtalk or whatever who can meet me online in the mornings, give me some direction for the day and then answer some questions here and there online throughout the day. Maybe a Python gury isn't necessary, maybe a person that know the language enough may be enough. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Sorting on multiple values, some ascending, some descending
Raymond Hettinger: The simplest way is to take advantage of sort-stability and do successive sorts. For example, to sort by a primary key ascending and a secondary key decending: L.sort(key=lambda r: r.secondary, reverse=True) L.sort(key=lambda r: r.primary) That's probably the faster and simpler way. The following solution is probably slow, memory-hungry, and it's not tested much so it may be buggy too, but it shows my idea, and bugs can be fixed: class Inverter: def __init__(self, item, reversed=False): self.item = item self.reversed = reversed def __cmp__(self, other): if self.reversed: return cmp(other.item, self.item) else: return cmp(self.item, other.item) data = [[1, a, b], [1, b, d], [1, d, a]] reverses = [True, False, False] print sorted(data, key=lambda subseq: map(Inverter, subseq, reverses)) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: static object
That looks like some kind of singleton. Why don't you use a module instead of a class? Another solution is to define your data as class attributes. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Sorting on multiple values, some ascending, some descending
dwelden wrote: L.sort(key=lambda r: r.secondary, reverse=True) L.sort(key=lambda r: r.primary) Excellent! That looks just like what I needed. Note that there is the (probably little used) operator.attrgetter() too, with that you can avoid the possibly slow lambda: L.sort(key=attrgetter(primary) operator.itemgetter(n) is when you have items that can be be accessed by index. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: code optimization (calc PI) / Full Code of PI calc in Python and C.
Michael M.: * The C is very fast, Python not. * Target: Do optimization, that Python runs nearly like C. Python can't be fast as C for that kind of programs. Note that your original C program gives less digits than the Python program. Your original takes about ~15.2 s on my PC. The following version (I have just cleaned it up a bit, probably it can be improved) needs about ~0.7 seconds with Psyco and ~5.7 s without (Psyco compilation time and Python startup times aren't included, if you include them the timings are ~0.94 s and ~5.96 s). Compiled with ShedSkin this program needs ~0.29 s (redirecting the output to a file, because ShedSkin printing is slow still). from time import clock def compute_pi(): pi = [] a = 1 b = d = e = g = 0 c = 5600 f = [2000] * (c + 4000 + 1) while c: d = 0 g = c * 2 b = c while b 1: d += f[b] * a g -= 1 f[b] = d % g d = d // g g -= 1 d *= b b -= 1 c -= 14 pi.append(%04d % int(e + d // a)) e = d % a return .join(pi) import psyco; psyco.bind(compute_pi) start_time = clock() print compute_pi() print Total time elapsed:, round(clock() - start_time, 2), s Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing more efficient code
Jon Harrop: I think this sort of functionality would be a welcome addition to Python. I don't know. Perhaps it can be written in Python? Pyparsing and beautifulsoup show that practically useful parsing modules can be done using Python alone too. Set module of Python 2.3, translated to C in Python 2.4 and improved in Python 2.5 shows that sometimes Python is fit to create prototypes that can be debugged, and whose API can be improved, and they can later translated to a faster language. But the decimal module shows that sometimes such translation can become a bit difficult. The re module shows that external modules too can be good enough compared of Perl/Ruby built-in regex syntax. Beside Python, various much faster languages may be used, like D, Pyrex, C, or your loved OCaml. Or theoretically even ShedSkin. I think D may be fit, the Pyd Python - D bridge is quite good, and it's improving. And D is reaching version 1.0. OCaml maybe can be used to produce Python compiled modules, so it can be a possibility too, but then very few people are able to maintain it, so maybe it's better to use a more common language (even D is more common, because its syntax is easy to grasp by C++ programmers). What is the easiest way to add such functionality to Python? I think implementation language isn't the main problem. I think the definition of the purpose and API are more difficult. Mathematica allows you to define rules that let the system chose the right function (among some with the same name) according to the input pattern or the kind of input. Guido has discussed some about something similar, the multimethods. There are some ways and tricks to add such capabilities to Python, but I don't think they are fast and reliable enough for real programs. Maybe Python 3.0 will change this some. If you want just to create something like a regular engine that works on lists, that contains matching rules, rewriting rules and calls to many user-defined functions, then I think you can do it with Python (with Psyco if you want) in few lines (a really basic RE on lists can be defined in about 20 lines, maybe a good system may can be built with 2000-1 lines), but I don't know how much useful it can be, maybe it can find some purpose (and maybe someone has already written such module). I think that is an excellent idea. Who will pay me? ;-) I don't know, probably no one. Most people don't even know how to use such pattern matching programming paradigm. For Python it may become just an experiment. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Iterate through list two items at a time
Few alternative solutions (other are possible), I usually use a variant of the first version, inside a partition function, the second variant is shorter when you don't have a handy partition() function and you don't want to import modules, and the forth one needs less memory when the data is very long: from itertools import izip, islice data = [1,2,3,4,5,6,7] for x1, x2 in (data[i:i+2] for i in xrange(0, len(data)/2*2, 2)): print x1, x2 for x1, x2 in zip(data[::2], data[1::2]): print x1, x2 for x1, x2 in izip(data[::2], data[1::2]): print x1, x2 for x1, x2 in izip(islice(data,0,None,2), islice(data,1,None,2)): print x1, x2 Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing more efficient code
Jon Harrop: OCaml's pattern matcher is very sophisticated and very fast. You'll probably shrink your code by several fold whilst also having it run a few orders of magnitude faster and having it statically checked, so it will be more reliable. You seem to forget some months of time to learn OCaml. And my Python programs seem reliable enough despite being unstatically checked :-) You might want to look at any language with pattern matching: OCaml, SML, Haskell, F#, Mathematica etc. Mathematica pattern matching is good, but Mathematica costs a lot of money (and you need some time to learn it, it's not an easy system). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing more efficient code
Jon Harrop: I think most people could pick up the core ideas in a day and start writing working programs. Probably I am not that intelligent, I probably need some months :-) But that language has many good sides, and one day I'll probably try to learn it a bit. Mathematica is expensive but learning to use pattern matching is much easier than learning how to write a pattern matcher and much less tedious than reimplementing it yourself all the time (which is exactly what the OP will end up doing). I see. This is a very old post of mine, at the bottom there are few notes about the Mathematica pattern matching syntax: http://groups.google.com/group/comp.lang.python/msg/93ce3e9a08f5e4c7 To avoid reimplementing it yourself all the time then maybe someone (you?) can try to write a good pattern matcher for sequences for CPython. With such system it may become less important to switch to a different language ;-) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Scaling pictures
Kajsa Anka: I found the Python Imaging Library but before I dive into that I would like to know if there is a better way of doing this. PIL is very fit for that. Note that it creates thumbnails already by itself, you can use that for bigger images too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: answers.py v0.0.1 - source
Marc 'BlackJack' Rintsch:
You give lot of comments and I agree with most of them.
One idiom to solve this is:
def __init__(self, answers=None):
self.answers = answers or dict()
I suggest to avoid such usage of or/and (expecially for newbies), that
is triky and can produce bugs, and to use a more explicit if:
def __init__(self, answers=None):
if answers is None:
self.answers = {}
else:
self.answers = answers
If you are sure you can use 2.5+ (the enclosing of the if is optional,
but helps readability):
def __init__(self, answers=None):
self.answers = ({} if answers is None else answers)
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
__getattr__ possible loop
I have tried this, with Psyco it segfaults, and with Python 2.5 (on Win) hangs the interpreter, is it possible to improve the situation? class T(object): def __getattr__(self, x): dir(self) #import psyco #psyco.full() T().method() (Probably dir calls __getattr__). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: __getattr__ possible loop
Maksim Kasimov: how to improve the situation depends on what do you expect to get by calling T().method() You are right, sorry for being cryptic. I think that's a kind of bug of Python (produced maybe by an infinite loop), so an improvement can be a traceback or some automatic breaking of that loop. Note that my problem comes from using Psyco that segfaults in that situation (if you have Psyco installed you can decomment two lines to see that). I think that using normal code Python+Psyco don't have to segfault, but I know this is tricky situation, so if no simple solutions can be found, then it's not a important thing and it can be ignored. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: loose methods : Smalltalk asPython
Steven D'Aprano: You can't modify the built-in classes. I'm not sure that it is a good idea to allow built-ins to be modified. When I see an int, I like the fact that I know what the int can do, and I don't have to worry about whether it has been modified by some piece of code elsewhere. I think it can be useful, there are many situations where you may need to change the behavior of built-in dicts, etc, but: - You need to keep code sorted and tidy to avoid problems. (This is true for most of Python code too today. For example ObjectPascal gives you less freedom to shoot yourself in the foot.) - Allowing the built-in objects to be dynamically modified like that may slow down the virtual machine some, if you don't design it quite carefully. Psyco can help here too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: some OT: how to solve this kind of problem in our program?
For people that will read the posts in the future, there is a little bug (it doesn't change the output of this program): items = alist[:] Has to be: alist = alist[:] Sorry, bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: some OT: how to solve this kind of problem in our program?
Paul McGuire: This little framework takes about 4.5 seconds, but with psyco, cuts down to about 1.3 seconds. st = time.time() result = [] for p in permutations(range(1,10)): aresult = func(p) if aresult is not None and aresult not in result: result.append(aresult) et=time.time() print 'time elapsed: %.4f s' % (et-st) for [[a0, a1, a2], [b0, b1, b2], [c0, c1, c2]] in result: print ' %0d %0d %0d %0d' % (a0, b0, c0, d0) print '--- + --- + --- = ---' print ' %0d%0d%0d%0d%0d%0d %0d' %(a1, a2, b1, b2, c1,c2, d1) print If you want to more speed, put long loops always inside functions, not inside the main body: def main(): st = clock() result = [] for p in permutations(range(1, 10)): aresult = func(p) if aresult is not None and aresult not in result: result.append(aresult) et = clock() print 'time elapsed: %.4f s' % (et-st) for [[a0, a1, a2], [b0, b1, b2], [c0, c1, c2]] in result: print ' %0d %0d %0d %0d' % (a0, b0, c0, d0) print '--- + --- + --- = ---' print ' %0d%0d%0d%0d%0d%0d %0d' %(a1, a2, b1, b2, c1,c2, d1) print main() If you have a bit of time you can test the speed of this code on your computer. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: some OT: how to solve this kind of problem in our program?
Using Psyco this version is much faster, you can test it on your PC compared to the other one (the whole running time, Psyco compilation too): Psyco is unable to speed up generator functions, so you have to return true lists. Giving the func to the permutation function, you can avoid lot of list copying and unpacking. try: import psyco psyco.full() except ImportError: pass d0, d1 = 1, 2 def func(p): a0,a1,a2,b0,b1,b2,c0,c1,c2 = p # do application evaluation here b1b2 = 10*b1+b2 a1a2 = 10*a1+a2 c1c2 = 10*c1+c2 if d1*a0*b1b2*c1c2 + d1*b0*a1a2*c1c2 + d1*c0*a1a2*b1b2 \ == d0*a1a2*b1b2*c1c2: return sorted( [[a0, a1, a2], [b0, b1, b2], [c0, c1, c2]] ) else: return None def accepted_permutations(alist, func): # func must return None for the unacceptable results # Algoritm from Phillip Paul Fuchs, modified result = [] items = alist[:] n = len(alist) p = range(n+1) i = 1 r = func(alist) if r is not None: result.append(r) while i n: p[i] -= 1 if i 1: j = p[i] else: j = 0 alist[j], alist[i] = alist[i], alist[j] r = func(alist) if r is not None: result.append(r) i = 1 while p[i] == 0: p[i] = i i += 1 return result def main(): result = [] for aresult in accepted_permutations(range(1, 10), func): if aresult not in result: result.append(aresult) [[a0, a1, a2], [b0, b1, b2], [c0, c1, c2]] = aresult print ' %0d %0d %0d %0d' % (a0, b0, c0, d0) print '--- + --- + --- = ---' print ' %0d%0d%0d%0d%0d%0d %0d'%(a1,a2,b1,b2,c1,c2,d1) print main() Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Multi-line docstrings
Duncan Booth: Not spuriously included: included by design, but sometimes annoying. Then it's a design decision I don't understand... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Decorator for Enforcing Argument Types
Chris wrote: I'm not sure if this has been done before, I think this is the best implementation: http://oakwinter.com/code/typecheck/ I have never used it, but it seems well done. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
array, a better shell
For array.array B means unsigned char, and such arrays accept to be initialized from (str) strings too, this is quite useful: from array import array a = array(B, hello) But it seems such capability isn't shared with the append: a.extend(hello) Traceback (most recent call last): File stdin, line 1, in module TypeError: an integer is required I like a lot the Python shell, it helps me in many different situations. I have tried some Python shells: - the vanilla one from DOS - the one from ActivePython IDE - the one from SPE - ipython from a DOS shell But none of them has what I'd like. ipython tries to be a better shell, but I don't want more complexity and more commands/tricks to remember, I want something more interactive, that's simpler and more powerful to use, and not something (much) more complex. Much less things, but the important ones. Beside the vanilla one, I end using the SPE one (because it has colours on Windows too, and it allows you to paste a piece of interactive shell with the leading , SPE removes them automatically. This is very handy). I have also used the shell of Mathematica. It's quite powerful and it can show graphics too inlined, but globally I don't like it fully because it makes editing small programs a pain (for me). Even if I globally don't like the Mathematica shell, it has a capability that I'd like to have in a basic Python shell too. (NOTE: often when dealing with GUI subtle details make a *big* difference, and it's not easy to use words to describe such interaction details.) This Mathematica shell allows you to edit small programs (like 1-15 lines of code) as input blocks, and later you can click on them and edit them. When you press shift-enter inside a block, that small program runs and its output goes just below it (and not at the end of the current shell log). All the Input Blocks can be edited and run again like that (an Input/Output number tag helps to keep things sorted enough). So it's a cross between a normal stupid shell that's essentially an I/O + visual log, and a bare-bone text editor that allows you to edit one script and run it too. Such capability probably needs a Tk/Wx/Gtk window... --- (Such interactive sessions can be saved too, and loaded again (they become complex documents), but such ability isn't necessary into a bare-bone shell that I am describing now. I am describing something as simple as possible). Beside that (new) basic shell capability I think I can appreciate two other capabilities: - Automatically saves the last 20 MBytes of textual input/output log inside a queue file. Material older than the last 20 MB is removed from the top. - Ability to break the computation (or the printing of VERY long things!) without exiting the shell and the kernel (this is from Mathematica too). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: array, a better shell
Steven D'Aprano: No you're not. You're describing a quite complicated shell. You're describing a hypothetical shell with features other actual shells don't have, so therefore it can't possibly be as simple as possible. You are right, it's not really simple, but: - It has just the basic functionality that I think is important. Many more features can be added, I too can list some of them, but I don't think they are much important. - It's very simple from the user point of view, because its usage requires no new commands to remember :-) (beside shift-enter or something similar to run a block). Thank you for listening, bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: array, a better shell
Duncan Booth: Later you can click on them and bring them back to the bottom of the input buffer for further editing (so no confusing output appearing out of order), I think that's worse, not better. You end with a messy final document (log), so finding things into it (during the editing too) is much more difficult. Your point was? My point is to suggest things that can improve the Python user experience, and productivity too. I try to help, with the hope to have something that I like more too. It's very difficult to describe subtle GUI functionalities using a texual description (expecially when you aren't using your native language). If you try the Mathematica shell you may find some differences I am talking about (even if I don't globally like the Mathematica shell). Editing an input block into idle feels different from editing a small script inside an editor, there are differences that make the user experience less good. (And beside that it seems I often have problems running IDLE on Win PCs with a firewall). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Shed Skin - Does it break any Python assumptions?
Jean-Paul Calderone: So yes, it seems that what ShedSkin supports is pretty distant from what a seasoned Python developer might expect, aside from syntactic constructs. At the moment SS doesn't allow to change the type of a variable during its lifetime, but SS is a moving target, maybe in the (far) future it will split it into two variables... I'm sure Mark likes to receive practical suggestions on how to improve SS to make it closer to Python. (And I think he likes to receive patches too, but usually it's better for people to start from simpler things, like speeding up a lib written with C++, etc). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Writing and reading variables to/from flat file
Geoffrey Clements:
readfile = open('prefs').readlines()
for line in readfile:
print line
eval(line)
print Basic
Instead of using eval, using a plain dict may be a better solution.
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: AI library
Gabriel Genellina: This is more stylish, but I prefer to use isxxx() or hasxxx() for functions that return booleans. Lisp-like languages allow the ending ? or !, I think Ruby allows the ending ? too (allowing it with Python may be positive). Mathematica usually uses an ending uppercase Q, like PrimeQ, sometimes I use the ending Q in Python too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: speed of python vs matlab.
Chao, you can also try Psyco, applied on functions, and when necessary using its metaclass too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: automatically grading small programming assignments
Brian Blais, just an idea. Create an online form to upload the tiny program(s). Such programs can be one for file. Then on your PC you can run a script that loads each of such programs, and runs a good series of tests, to test their quality... Such tests can be about all things, speed, coding quality, that the results are correct, etc. You can even put some way to autenticate students... (BTW, good students don't cheat. Cheating isn't a good way to learn to program, and they probably know this). Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: merits of Lisp vs Python
Andrea GriffiniIs this worth investigation or it has already been suggested/tried ? Recently some people around the net have discussed about similar ideas as a possible way to speed up Ruby interpreters a lot. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: len() and PEP 3000
Colin J. Williams: Why not replace the __len__ method with a len property for strings, lists, tuples, dictionaries etc. __len__ is not very special and the property len eliminates the redundant parentheses. You mean something like: ab.len, [1, 2, 3].len (2, 3) In the given page Guido says: I didn't want these special operations to use ordinary method names, because then pre-existing classes, or classes written by users without an encyclopedic memory for all the special methods, would be liable to accidentally define operations they didn't mean to implement, with possibly disastrous consequences. I think it's easy enough to remember a len attribute, you use it all the time. many functions are defined in terms of informal interfaces; for example, reversed works on anything that supports random access to items and has a known length. In practice, implementing things like max, sum, map, any, in and others, as built-in functions and operators is actually less code than implementing them as methods for each and every type that needs to support them. I agree, but I think some exceptions can be made for len attribute, and for .copy(), .deepcopy(), and maybe for few other general methods. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: merits of Lisp vs Python
[EMAIL PROTECTED]: Sorry, I missed something here. Why do you need a release to have these sorts of things? Can't you just expand the language via macros to create whatever facility of this sort you need... Oh, sorry. You CAN'T expand the language Too bad. I guess waiting for Guido to figure out what Fits Your Mind is part of The Python Way. There are few programmers that are very intelligent and know how very well how to design languages, so they can invent such things. They are able to manage and appreciate the more freedom Lisp gives them. But *most* other programmers aren't intelligent enough for that, or they don't know enough language design to produce something working or something nice. So maybe Lisp is for the more intelligent people, or for really difficult programs (or for dynamic and flexible programs that have to run quite faster than Python+Psyco code), where having a bit simpler language isn't an advantage. All the other people use Python that contains things already well designed for them, such language constructs are the same for everybody, they are standards of the language, like generators, so reading each other code is simpler and faster too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: merits of Lisp vs Python
[EMAIL PROTECTED]: Sorry, I missed something here. Why do you need a release to have these sorts of things? Can't you just expand the language via macros to create whatever facility of this sort you need... Oh, sorry. You CAN'T expand the language Too bad. I guess waiting for Guido to figure out what Fits Your Mind is part of The Python Way. There are few programmers that are very intelligent and know how very well how to design languages, so they can invent such things. They are able to manage and appreciate the more freedom Lisp gives them. But *most* other programmers aren't intelligent enough for that, or they don't know enough language design to produce something working or something nice. So maybe Lisp is for the more intelligent people, or for really difficult programs (or for dynamic and flexible programs that have to run quite faster than Python+Psyco code), where having a bit simpler language isn't an advantage. All the other people use Python that contains things already well designed for them, such language constructs are the same for everybody, they are standards of the language, like generators, so reading each other code is simpler and faster too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Factory pattern implementation in Python
Dennis Lee Bieber:
Presuming the event x is a type code I'd just set up a list of functions:
Then create a dictionary of them, keyed by the event x code
processors = { 1 : process_1,
2 : process_2,
x : process_x }
Just a dict of functions was my solution too, I think avoiding more
complex solutions is positive.
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: Remarkable results with psyco and sieve of Eratosthenes
George Sakkis: You can also save an attribute lookup for append; just add append = primes.append outside of the loop and replace primes.append(x) with append(x) That should cut down a few fractions of second. We were talking about Psyco, and I think with Psyco (just released for Py 2.5, BTW) such tricks are less useful. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: best way to align words?
Robert R.: i would like to write a piece of code to help me to align some sequence of words and suggest me the ordered common subwords of them [...] a trouble i have if when having many different strings my results tend to be nothing while i still would like to have one of the, or maybe, all the best matches. This is my first solution try, surely there are faster, shorter, better solutions... from collections import defaultdict from itertools import chain from graph import Graph # http://sourceforge.net/projects/pynetwork/ def commonOrdered(*strings): lists = [[w for w in string.lower().split() if w.isalpha()] for string in strings] freqs = defaultdict(int) for w in chain(*lists): freqs[w] += 1 g = Graph() for words in lists: g.addPath(words) len_strings = len(strings) return [w for w in g.toposort() if freqs[w]==len_strings] s0 = this is an example of a thing i would like to have s1 = another example of something else i would like to have s2 = 'and this is another example but of something ; now i would still like to have' print commonOrdered(s0, s1, s2) It creates a graph with the paths of words, then sorts the graph topologically, then takes only the words of the sorting that are present in all the original strings. With a bit of work the code can be used if it contains words like example instead of example . An xtoposort method too can be added to the Graph class... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: best way to align words?
This is my first solution try, surely there are faster, shorter, better solutions... It creates a graph with the paths of words, then sorts the graph topologically, Beside possible inefficiencies, this solution breaks if words aren't in the correct order, the topological sort can't work... I'll have to think about better solutions, if possible. Sorry, bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Modifying every alternate element of a sequence
Leo Kislov: input[1::2] = [-item for item in input[1::2]] If you don't want to do it in-place, just make a copy: wanted = input[:] wanted[1::2] = [-item for item in wanted[1::2]] Very nice solution. I have tried few versions like: from itertools import imap, islice from operator import neg 1) data[1::2] = [-el for el in data[1::2]] 2) data[1::2] = map(neg, data[1::2]) 3) data[1::2] = imap(neg, data[1::2]) 4) data[1::2] = map(neg, islice(data, 1, None, 2)) 5) etc. With Python 2.5 it seems that the n.2 (map + slicing) is the faster. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: problem about list indexing
hollowspook: how about indexing 1-7, 10 [range(1:8),10] will generate [[1, 2, 3, 4, 5, 6, 7], 10], instead of [1, 2, 3, 4, 5, 6, 7, 10] (Note that range(1:8) is a syntax error). You can join and extend lists as you like: range(1, 8) + [10] [1, 2, 3, 4, 5, 6, 7, 10] See also the list.append and list.extend methods too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Graph Data Structures
Szabolcs Nagy: i haven't read your code, but there are many graph implementations in python. in case you haven't found these yet: http://wiki.python.org/moin/PythonGraphApi if you only want to do some analysis i think you need this one (as it's pretty complete and simple): https://networkx.lanl.gov/ i also recommend Guido's essay to read: http://www.python.org/doc/essays/graphs.html I can also suggest my one: http://sourceforge.net/projects/pynetwork/ And boost graph bindings for Python, quite fast: http://www.osl.iu.edu/~dgregor/bgl-python/ Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: python gaining popularity according to a study
[EMAIL PROTECTED]: Python is the 7th most commonly used language, up from 8th. The only one gaining ground besides VB in the top 10. It also shows that Ruby is gaining even more, and D is (gladly) growing too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: text file parsing (awk - python)
Peter Otten, your solution is very nice, it uses groupby splitting on empty lines, so it doesn't need to read the whole files into memory. But Daniel Nogradi says: But the names of the fields (node, x, y) keeps changing from file to file, even their number is not fixed, sometimes it is (node, x, y, z). Your version with the converters dict fails to convert the number of node, z fields, etc. (generally using such converters dict is an elegant solution, it allows to define string, float, etc fields): converters = dict( x=int, y=int ) I have created a version with a RE, but it's probably too much rigid, it doesn't handle files with the z field, etc: data = node 10 y 1 x -1 node 11 x -2 y 1 z 5 node 12 x -3 y 1 z 6 import re unpack = re.compile(r(\D+) \s+ ([-+]? \d+) \s+ * 3, re.VERBOSE) result = [] for obj in unpack.finditer(data): block = obj.groups() d = dict((block[i], int(block[i+1])) for i in xrange(0, 6, 2)) result.append(d) print result So I have just modified and simplified your quite nice solution (I have removed the pprint, but it's the same): def open(filename): from cStringIO import StringIO return StringIO(data) from itertools import groupby records = [] for empty, record in groupby(open(records.txt), key=str.isspace): if not empty: pairs = ([k, int(v)] for k,v in map(str.split, record)) records.append(dict(pairs)) print records Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: regex problem
line is am trying to match is
1959400|Q2BYK3|Q2BYK3_9GAMM Hypothetical outer membra29.90.00011 1
regex i have written is
re.compile
(r'(\d+?)\|((P|O|Q)\w{5})\|\w{3,6}\_\w{3,5}\s+?.{25}\s{3}(\d+?\.\d)\s+?(\d\.\d+?)')
I am trying to extract 0.0011 value from the above line.
why doesnt it match the group(4) item of the match ?
any idea whats wrong with it ?
I am not expert about REs yet, but I suggest you to use the re.VERBOSE
and split your RE in parts, like this:
example = re.compile(r^ \s* # must start at the beginning +
optional whitespaces
( [\[\(] ) # Group 1: opening bracket
\s*# optional whitespaces
( [-+]? \d+ ) # Group 2: first number
\s* , \s* # optional space + comma +
optional whitespaces
( [-+]? \d+ ) # Group 3: second number
\s*# optional whitespaces
( [\)\]] ) # Group 4: closing bracket
\s* $ # optional whitespaces + must
end at the end
, flags=re.VERBOSE)
This way you can debug and mantain it much better.
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Abelson and Python
While studying the SICP video lectures I have to twist my mind some to completely understand the lessons. I implement the programs shown there in both Python and Scheme, and I find the Python implementations simpler to write (but it's not a fair comparison because I know very little Scheme still). Now some things are changing: http://lambda-the-ultimate.org/node/1840 The MIT is going to change its curriculum structure that was famous for teaching Scheme in introductory courses. One force behind the reform is no one else than Harold Abelson, famous for his marvelous Scheme opus SICP. The first four weeks of C1 will be a lot like the first four weeks of 6.001, Abelson said. The difference is that programming will be done in Python and not Scheme. Someone is translating the SIPC programs in Python too: http://www.codepoetics.com/wiki/index.php?title=Topics:SICP_in_other_languages#Python With slides and code from lectures: http://courses.csail.mit.edu/6.01/ http://courses.csail.mit.edu/6.01/fall06/calendar.html I like Scheme a bit too, but I consider this is a small victory for Python :-) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Abelson and Python
[EMAIL PROTECTED]: No surprise to anyone who's ever tried to use MIT Scheme. Be careful, such assertions are often flamebait. I am using DrPython (I think they were using it at MIT too lately), and it is very very good IDE, it produces executables on the fly, it has a visual debugger with some nice graphical things, it manages graphics, and it has something that I have never seen in Python: it manages a hierarchy of simpler Scheme languages, useful to learn for students. Probably something similar may be useful to learn Python too (such stripped down Python versions can forbid things like def foo(x=[]): ...). I am appreciating Scheme a bit because of such very good editor that makes things possible for newbies of the language too. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: AVL Balancing
scbauer wrote: This is one of the errors that im getting Traceback (most recent call last): File pyshell#49, line 1, in module t.insert(5) File /Users/stevenbauer/Desktop/AVL.py, line 68, in insert stack.append(current) NameError: global name 'stack' is not defined def __init__(self): self.__root = None self.__stack=[] stack = self.__stack def addNode(self, data): . stack.append(current) Some partial suggestions: - Why do you use __? Use them only when necessary. My suggestion is to use: self._stack = [] - What's the purpose of stack = self.__stack ? It it does nothing there. - stack.append(current) can't find the stack name, because it's not present in this scope, it's present into the __init__ only. My suggestion is to use the instance attribute name. You can probably fix that problem now. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Abelson and Python
[EMAIL PROTECTED]: Haven't heard of that one, although I've got DrScheme. Right, sorry, I meant that one :-) I find that hierarchy extremely annoying. I don't see the need for it. I never use OOP in Python yet there's no need for me to have a stripped down version, I just don't use it. But Python too contains some things that can bite a newbie of the language. (BTW, in some situations I too have had to access to the full Scheme language, so you may be partially right). But those are implementation details, which you can't avoid. A bad implementation spoils a language even if the language itself is fabulous. And a good implementation like DrScheme makes the Scheme language usable too :-) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Abelson and Python
markscottwright: I love Python as much as the next guy, but I just don't see how SICP can be done in Python. The contents of the course are probably different, they work on robotics... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: changing list items
A possibile first solution: alist = ['a','b','c','e','d','f'] inf, sup = 2, 4 alist[inf:sup] = [t] * (sup - inf) alist ['a', 'b', 't', 't', 'd', 'f'] Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Abelson and Python
Paddy: Is the MIT course syndicated to Universities around America or something? (Is your name pronounced Beer-owe-file, or Bear-oh-fi-lee, I don't know. I too have heard about the MIT course changing to Python elsewhere and wanted to know why it was talked about so much? I don't know why others have talked so much about it, I have posted the links here because those CS courses were very good, and probably originally they seemed more like science finction. The most important link I have shown is among the last ones, to the slides of the lectures, that can probably be used to learn some other CS. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
dict.reserve and other tricks
I have started doing practice creating C extensions for CPython, so
here are two ideas I have had, possibly useless.
If you keep adding elements to a CPython dict/set, it periodically
rebuilds itself. So maybe dict.reserve(n) and a set.reserve(n) methods
may help, reserving enough (empty) memory for about n *distinct* keys
the programmer wants to add to the dict/set in a short future. I have
seen that the the C API of the dicts doesn't allow this, and I don't
know if this can be implemented modifying the dicts a bit. Do you think
this may be useful?
---
Sometime I need to remove some elements from dicts/sets. If I know a
rule that tells what elements have to be removed I can use code like
this:
import itertools
def filterset(predicate, inset):
return set(itertools.ifilter(predicate, inset))
print filterset(lambda x: x 1, set(range(10)))
Output:
set([1, 3, 9, 5, 7])
And a similar one to filter dicts that works with a predicate(key,
val).
Most of the times such functions are good enough, but sometimes the
dicts are big, so to reduce memory used I remove keys in place:
def filterdict(pred, indict):
todel = [k for k,v in indict.iteritems() if not pred(k,v)]
for key in todel:
del indict[key]
d = dict.fromkeys(xrange(8), 0)
print d
filterdict(lambda k,v: k 1, d)
print d
# Output:
# {0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0}
# {1: 0, 3: 0, 5: 0, 7: 0}
But doing the same thing while iterating on the dict may be faster and
use even less memory.
This iterationdeletion capability is probably not safe enough to be
used inside Python programs, but a compiled C module with a function
that works like that filterdict (and deletes while iterating) may be
created, and its use is safe from Python programs.
The C++ STL API of maps allows to delete items while iterating on the
map (but probably it has to be done only when necessary, because it may
be a source for bugs):
typedef mapint, string isMap;
typedef isMap::value_type isValType;
typedef isMap::iterator isMapItor;
void main() {
isMap m;
... // items added to m
for(isMapItor itr = m.begin(); itr != m.end();) {
if(itr-second == somestring)
m.erase(itr++);
else
++itr;
}
The Python/C API, 7.4.1 Dictionary Objects, says that's not possible
with CPython dicts:
The dictionary p should not be mutated during iteration. It is safe (since
Python 2.1) to modify the values of the keys as you iterate over the
dictionary, but only so long as the set of keys does not change.
Do you think it may be useful to create to create such C filterdict
function that deletes while iterating? (To create such function it
probably has to bypass the CPython dict API).
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict.reserve and other tricks
Thank you for the answers Terry Reedy and Klaas. Since you are writing extensions, you can create a built-in subclass of dict to experiment with. I presume the 2.5 default dict should be a model. That way it's doable, but I think it's of limited use too; I'd like to remove elements from arbitrary (normal) dicts. Klaas: Well, you can reduce the memory usage to virtually nothing by using a generator expression rather than list comprehension. Are you sure? I don't think so. Can you show a little example? As you know this can't work: def filterdict(pred, indict): todel = (k for k,v in indict.iteritems() if not pred(k,v)) for key in todel: del indict[key] d = dict.fromkeys(xrange(8), 0) print d filterdict(lambda k,v: k 1, d) print d (You can remove elements from the dict and put them into a list, and then put back the elements into the dict, this is probably the only way I know of *theoretically* keeping the memory used about the same with Python. In practice the less memory consuming version may be the filterdict I have shown). arbitrary python code can be executed by __hash__ and deleting (DECREF) python objects. I am starting to understand. I'll think more about this. Thank you, bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: simple way to un-nest (flatten?) list
djc: As it is possible that the tuples will not always be the same word in variant cases result = sum(r.values(), ()) will do fine and is as simple as I suspected the answer would be. It is simple, but I suggest you to take a look at the speed of that part of your code into your program. With this you can see the difference: from time import clock d = dict((i,range(300)) for i in xrange(300)) t = clock() r1 = sum(d.values(), []) print clock() - t t = clock() r2 = [] for v in d.values(): r2.extend(v) print clock() - t assert r1 == r2 Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: string to list of numbers conversion
[EMAIL PROTECTED] wrote: Hi, I have a string '((1,2), (3,4))' and I want to convert this into a python tuple of numbers. But I do not want to use eval() because I do not want to execute any code in that string and limit it to list of numbers. Is there any alternative way? This is a possibile solution, no input errors are taken into account: s = '((1,2), (3,4), (-5,9.2))' from string import maketrans tab = maketrans((), , *4) s.translate(tab) ' 1 23 4-5 9.2 ' l = s.translate(tab).split() l ['1', '2', '3', '4', '-5', '9.2'] l2 = map(float, l) l2 [1.0, 2.0, 3.0, 4.0, -5.0, 9.1993] # This is partition(l2, 2) [l2[i:i+2] for i in xrange(0, len(l2), 2)] [[1.0, 2.0], [3.0, 4.0], [-5.0, 9.1993]] Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Defaultdict and speed
Klaas wrote:
Benchmarks?
There is one (fixed in a succesive post) in the original thread I was
referring to:
http://groups.google.com/group/it.comp.lang.python/browse_thread/thread/aff60c644969f9b/
If you want I can give more of them (and a bit less silly, with strings
too, etc).
def ddict(n):
t = clock()
d = defaultdict(int)
for i in xrange(n):
d[i] += 1
print round(clock()-t, 2)
def ndict(n):
t = clock()
d = {}
for i in xrange(n):
if i in d:
d[i] += 1
else:
d[i] = 1
print round(clock()-t, 2)
ddict(30)
ndict(30)
(and slowing down other uses of the class)
All it has to do is to cheek if the default_factory is an int, it's
just an if done only once, so I don't think it slows down the other
cases significantly.
especially when the faster alternative is so easy to code.
The faster alternative is easy to create, but the best faster
alternative can't be coded, because if you code it in Python you need
two hash accesses, while the defaultdict can require only one of them:
if n in d:
d[n] += 1
else:
d[n] = 1
If that performance difference matters,
With Python it's usually difficult to tell if some performance
difference matters. Probably in some programs it may matter, but in
most other programs it doesn't matter. This is probably true for all
the performance tweaks I may invent in the future too.
you would likely find more fruitful
gains in coding it in c, using PyDict_SET
I've just started creating a C lib for related purposes, I'd like to
show it to you all on c.l.p, but first I have to find a place to put it
on :-) (It's not easy to find a suitable place, it's a python + c +
pyd, and it's mostly an exercise).
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Defaultdict and speed
This post sums some things I have written in another Python newsgroup.
More than 40% of the times I use defaultdict like this, to count
things:
from collections import defaultdict as DD
s = abracadabra
d = DD(int)
for c in s: d[c] += 1
...
d
defaultdict(type 'int', {'a': 5, 'r': 2, 'b': 2, 'c': 1, 'd': 1})
But I have seen that if keys are quite sparse, and int() becomes called
too much often, then code like this is faster:
d = {}
for c in s:
... if c in d: d[c] += 1
... else: d[c] = 1
...
d
{'a': 5, 'r': 2, 'b': 2, 'c': 1, 'd': 1}
So to improve the speed for such special but common situation, the
defaultdict can manage the case with default_factory=int in a different
and faster way.
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: other ways to check for type 'function'?
elderic: 1.: -- sort of clumsy and discouraged by the docs as far as I read import types type(f) is types.FunctionType What's the problem with this? from types import FunctionType if isinstance(f, FunctionType): ... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Python in sci/tech applications
mattf: 3) -There's a problem with development under Windows. It's possibile to compile Python with MinGW, and to create extensions with it. So some site can host a single zip file that contains both MinGW and Python compiled with it, all ready and set. A person not much expert can then create compiled small extensions in a short time without too much complexities. (Or maybe the D language can be packed into that, instead of MinGW, to do similar things. D can be a good language used with Python) But then probably PIL, scipy, etc have to be compiled again for such alternative official or semi-official Python distribution. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Python in sci/tech applications
Robert Kern: We distribute mingw set up to do this with our Enthought Edition Python distribution. http://code.enthought.com/enthon/ Sorry, maybe I'm blind but I don't see MinGW listed in that page... Maybe it's included but not listed... Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Python in sci/tech applications
Fredrik Lundh: last time I tried, it took me 20 minutes from that I typed mingw into google until I had built and tested my first non-trivial extension. your milage may vary. But probably before those 20 minutes there is a lot of time of experience of yours with CPython sources, other compilers, and so on, so for most people this timing comparison just means It can be done :-) Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Style for modules with lots of constants
Neil Cerutti:
scriptref = glk.fileref_create_by_prompt('Transcript+TextMode',
'WriteAppend', 0)
That + sign seems useless. A space looks enough to me. The functions
can accept case-agnostic strings and ignore spaces inside them.
Example:
('transcript textmode ', 'writeappend', 0)
Parsing the combinable bitfield contants might be slowish, though.
I think you have to test if this is true for your situation. Management
of interned strings is probably fast enough (compared to the rest of
things done by the Python interpreter) for many purposes.
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
Re: Style for modules with lots of constants
Rob Williscroft: This is nice, but you can cut down on some of the cruft: class Constants( object ): pass Constants.RIGHT = 0 Constants.LEFT = 1 ## client code ... print Constants.LEFT Another possibility is to define such constants as strings instead of integers: _allflags = (left, right, # other constants... # other constants... ) class Flags(): pass for _flag in _allflags: setattr(Flags, _flag, _flag) #print Flags.__dict__ Then the user can use Flags.left+ +Flags , or just the left string. Then maybe for each attribute like the left constant, a left_doc attribute can be added to Flags with a textual explanation of the meaning of the left constant. If your module has a short name, then maybe it's better to just use such string constants as modulename.constant instead of Flags.constant, using globals()[_flag] = _flag Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
