Re: Refactor/Rewrite Perl code in Python
Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
On Sun, Jul 24, 2011 at 7:29 PM, Shashwat Anand anand.shash...@gmail.com
wrote:
How do I start ?
The idea is to rewrite module by module.
But how to make sure code doesn't break ?
By testing it.
Read up on test driven development.
At this point, you have this:
Perl modules: A, B, C, D
Python modules: none
Python tests: none
Now, before you rewrite each Perl module in Python, first write a good,
comprehension test suite in Python for that module. You need to have tests
for each function and method. Test that they do the right thing for both
good data and bad data. If you have functional requirements for the Perl
modules, use that as your reference, otherwise use the Perl code as the
reference.
For example, this might be a basic test suite for the len() built-in
function:
for empty in ([], {}, (), set([]), ):
if len(empty) != 0:
raise AssertionError('empty object gives non-zero len')
for n in range(1, 5):
if len(x*n) != n:
raise AssertionError('failure for string')
for kind in (list, tuple, set):
obj = kind([None]*n)
if len(obj) != n:
raise AssertionError('failure for %s' % obj)
if len({'a': 1, 'b': None, 42: 'spam'}) != 3:
raise AssertionError('failure for dict')
for bad_obj in (23, None, object(), 165.0, True):
try:
len(bad_obj)
except TypeError:
# Test passes!
pass
else:
# No exception means len() fails!
raise AssertionError('failed test')
Multiply that by *every* function and method in the module, and you have a
moderately good test suite for module A.
(You may want to learn about the unittest module, which will help.)
Now you have:
Perl modules: A, B, C, D
Python modules: none
Python tests: test_A
Now re-write the Python module, and test it against the test suite. If it
fails, fix the failures. Repeat until it passes, and you have:
Perl modules: A, B, C, D
Python modules: A
Python tests: test_A
Now you can be confident that Python A does everything that Perl A does.
Possibly *better* than the Perl module, since if you don't have a test
suite for it, it probably has many hidden bugs.
Continue in this way with the rest of the modules. At the end, you will have
a full test suite against the entire collection of modules.
A very sane approach.
I currently support a large legacy application written in C++ by
extending it with Python. We managed to do this by wrapping the legacy
application with a Python interface using the Boost::Python libraries.
It has allowed us to embrace and extend the legacy code and replace bits
of it one piece at a time while keeping the whole application running.
Now I am not aware of any Python bindings to the Perl interpreter, but
if there is some FFI library that makes it possible this approach might
be viable for your project.
The trade-off of this approach is the extra complexity of maintaining
another interface in your code. The beneift is that you can avoid
re-writing a large amount of code up front. This works particularly
well when the legacy system has a rather large user base and you don't
have a robust test suite for the legacy code.
One book I found particularly useful:
http://www.amazon.ca/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052
hth
--
http://mail.python.org/mailman/listinfo/python-list
Re: Functional style programming in python: what will you talk about if you have an hour on this topic?
Anthony Kong anthony.hw.k...@gmail.com writes:
(My post did not appear in the mailing list, so this is my second try.
Apology if it ends up posted twice)
Hi, all,
If you have read my previous posts to the group, you probably have some idea
why I asked this question.
I am giving a few presentations on python to my colleagues who are mainly
java developers and starting to pick up python at work.
personal opinion
So I have picked this topic for one of my presentation. It is because
functional programming technique is one of my favorite in my bag of python
trick. It also takes me to the rabbit hole of the functional programming
world, which is vastly more interesting than the conventional procedural/OO
languages.
/personal opinion
I think I will go through the following items:
itertools module
functools module
concept of currying ('partial')
I would therefore want to ask your input e.g.
Is there any good example to illustrate the concept?
What is the most important features you think I should cover?
What will happen if you overdo it?
Cheers
You might also want to cover gotchas like Python's references.
If you have the time I'd introduce weakref too.
--
http://mail.python.org/mailman/listinfo/python-list
Re: python interview quuestions
James Mills prolo...@shortcircuit.net.au writes: On Sat, Aug 7, 2010 at 4:32 AM, Tim Chase python.l...@tim.thechases.com wrote: I would like to aquint myself with Python Interview questions This came up a while ago: http://www.mail-archive.com/python-list@python.org/msg168961.html Most of that thread is still relevant (perhaps throw in some py3l questions too) A common thing you can do in interviews is ask your interviewee to write (in Python) a solution to the FizzBuzz problem. Any good competent Python programmer should be able to do this in 5-10mins (5 if you're good). cheers james Fizzbuzz is annoying in interviews. I've never worked at a job where I was under a timer while a group of people sat across from me and scrutinized everything I was doing. I don't see how it can honestly tell you anything useful about the person you're interviewing either. Do you really think that what you assume about the interviewee based on characteristics you can infer from their solution to be really, honestly true? They might even completely bomb the solution and still be a brilliant programmer, but you'll never know that if you trust this simple fizzbuzz test. I've been in those interviews on both sides of the table. Neither side was a good experience. If a test is necessary, make it a take-home or demand source code if they have it. Read their code and judge for yourself the quality of their work. Any questions in an interview should be the usual get to know you type stuff. What was the most difficult challenge you've faced on the job? How did you respond? That sort of thing. -- http://mail.python.org/mailman/listinfo/python-list
Re: List of lists surprising behaviour
candide cand...@free.invalid writes: Let's the following code : t=[[0]*2]*3 t [[0, 0], [0, 0], [0, 0]] t[0][0]=1 t [[1, 0], [1, 0], [1, 0]] Rather surprising, isn't it ? Not at all, actually. I'd be surprised if the multiplication operator was aware of object constructors. Even arrays are objects in Python. Should the multiplication operator know how to instantiate three arrays from a single array instance? What about an instance of a user-defined class? So I suppose all the subarrays reférence the same array : id(t[0]), id(t[1]), id(t[2]) (3077445996L, 3077445996L, 3077445996L) As they should. So what is the right way to initialize to 0 a 2D array ? Is that way correct : t=[[0 for _ in range(2)] for _ in range(3)] It seems there is no more trouble now : t [[0, 0], [0, 0], [0, 0]] t[0][0]=1 t [[1, 0], [0, 0], [0, 0]] Correct ? 2d_zero_vector = lambda len: [[0, 0] for _ in range(len)] t = 2d_zero_vector(3) print t [[0, 0], [0, 0], [0, 0]] t[0][0] = 1 print t [[1, 0], [0, 0], [0, 0], [0, 0]] (Of course, if you're doing matrix math you'll probably want to work with numpy which has a function for doing just this) -- http://mail.python.org/mailman/listinfo/python-list
Re: Can someone please make it more pythonic or better?
Oltmans rolf.oltm...@gmail.com writes:
Greetings Python superstars,
I've a directory structure like following
tests /
__init__.py
testfile.py
testfile.py contains following code
import unittest
class Calculator(unittest.TestCase):
def test_add(self):
print 'just add'
def test_divide(self):
print 'diviide'
def test_multiply(self):
print 'mul'
class Car(unittest.TestCase):
def test_start(self):
print 'start'
def test_move_right(self):
print 'move right'
def test_move_left(self):
print 'move left'
def test_stop(self):
print 'stop'
Now give the following user-input I want to get all test-names.
user-input = tests.testfile (get all test-names from all
unittest.TestCase derived classes in test.testfile)
user-input = tests.testfile.Car (get all test-names from the Car
class)
user-input = tests.testfile.Cacr.test_stop
and I'm doing it this the following way and I really think there has
to be more readable, more pythonic and more possibly short way to do
it
import unittest
import sys
import inspect
def get_test_names(full_name,module):
name = full_name.split('.')
loader = unittest.TestLoader()
if len(name) == 4:
return full_name
elif len(name) == 3:
exec from %s.%s import %s %(module,name[1],name[2])
return loader.getTestCaseNames(eval(name[2]))
elif len(name) == 2:
exec 'from %s import %s' % (module,name[1])
tests = []
for _name, obj in inspect.getmembers(sys.modules[full_name]):
if inspect.isclass(obj) and
issubclass(obj,unittest.TestCase):
exec from %s.%s import %s %
(module,name[1],obj.__name__)
tests.append(loader.getTestCaseNames(obj))
return tests
if __name__ == __main__:
input = tests.testfile
module = input.split('.')[0]
_tests = get_test_names(input,module)
print _tests
So guys, can you kindly point me to a more Pythonic, more readable and
possible more short way to acheive this? I will really appreciate any
help. Many thanks in advance.
First of all, exec is bad if it's going to be processing user input.
You might want to:
help(__import__)
It will give you an idea on how to hook into python's import machinery
for tasks such as this.
You could also modify the function's arglist to remove the
string-splitting you're doing. Those magic numbers stick out a bit.
One can understand what they're for after reading the code in this case,
but it's not quite necessary if you make a keyword argument for package
names you can pass into the 'fromlist' argument in __import__.
ie:
def get_test_names(module_name, packagelist=[]):
...
hth,
j_king
Best regards,
Oltmans
--
http://mail.python.org/mailman/listinfo/python-list
Re: Overcoming python performance penalty for multicore CPU
Paul Rubin no.em...@nospam.invalid writes: Stefan Behnel stefan...@behnel.de writes: Well, if multi-core performance is so important here, then there's a pretty simple thing the OP can do: switch to lxml. http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/ Well, lxml is uses libxml2, a fast XML parser written in C, but AFAIK it only works on well-formed XML. The point of Beautiful Soup is that it works on all kinds of garbage hand-written legacy HTML with mismatched tags and other sorts of errors. Beautiful Soup is slower because it's full of special cases and hacks for that reason, and it is written in Python. Writing something that complex in C to handle so much potentially malicious input would be quite a lot of work to write at all, and very difficult to ensure was really safe. Look at the many browser vulnerabilities we've seen over the years due to that sort of problem, for example. But, for web crawling, you really do need to handle the messy and wrong HTML properly. If the difference is great enough, you might get a benefit from analyzing all pages with lxml and throwing invalid pages into a bucket for later processing with BeautifulSoup. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Wed, 23 Dec 2009 10:55:19 -0500, J Kenneth King wrote: Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Mon, 21 Dec 2009 11:44:29 -0500, J Kenneth King wrote: A programmer that lacks critical thinking is a bad programmer. The language they use has no bearing on such human facilities. That's nonsense, and I can demonstrate it by reference to a single programming language, namely Python. For many years, Python had no ternary if operator: [...] But did the lack of ternary encourage Raymond to become a bad programmer? No, but Raymond started off in a position of being an excellent programmer. A single buggy idiom lead him to be slightly-less excellent than he otherwise would have been. How many buggy idioms would it take to lead him to become a mediocre coder, if he was unable to change languages? Because Python is generally an excellent language, the harm done by one or two misfeatures is minor. But less excellent languages encourage coding styles, techniques and idioms that encourage the programmer to write poor code: either complicated, baroque, unreadable code; or slow inefficient code; or buggy code. To avoid starting a flame war, I will avoid mentioning PHP. *cough* Sometimes you know what you need to do to write non-buggy code, but because covering all the corners are just Too Damn Hard in a certain language, you simply lower your expectations. Error checking is tedious and hard to get right in some languages, like C and Pascal, and hence even good programmers can miss some errors. Different languages encourage different mind-sets in the programmer: C encourages the coder to think at the low level of pointers and addresses, and primarily about machine efficiency; Java encourages the use of big object hierarchies and design patterns (it's hard to write lightweight code in Java, so everything turns into heavyweight code); Perl encourages cleverness and code-golf (writing a program in as few lines or characters as possible); Haskell and Lisp encourage a heavily abstract approach that often requires an elite coder to follow; Forth encourages you to think like Yoda. If anyone continues to follow bad idioms without questioning their usefulness from time to time, I'd question their ability as a programmer. Critical thinking is important. Which is why good programs can be written in PHP, Forth, Lisp, Perl, and anything else. However, if a programmer thinks the only language they will ever need to know is BF, they have a serious screw loose. ;) [...] Good tools make all the difference in the world, I'm not arguing that. You appear to be arguing against that. Maybe you need to reconsider my arguments. It takes a good programmer to recognize the values and trade-offs of the tools they work with. Ignorance is not an excuse to blame the language. It's too easy to say, Well Perl sucks because it encourages you to be a bad programmer because it has all these features that let you shoot yourself in the foot. In reality, lots of really great programs are written in Perl all the time and some very smart people write them. It just so happens that in hands of the educated, those very features are useful in certain cases. Python doesn't encourage you to be a better programmer. It just enforces particular idioms and conventions in its design. As long as the ignorant programmer follows them they should be better off. Yet if they are ignorant, no amount of encouragement will get them to think critically about Python and find bugs in it. They will have to rely on the community of developers to do that thinking for them. Just that the tools don't use us; we use them. Nobody said that tools use us. But it is being suggested that they influence our thinking. Pretty smart thing for a language to be able to do. Programming in Python doesn't instantly make me a better programmer. No, not instantly, but I would argue that after many years of coding in Python you will be a better programmer than after the same number of years of coding in PHP or Basic. And my argument is that the human element is what will determine who is better. There are good programmers who can program in PHP. Some of the biggest websites on the Internet are programmed in it. And like any language I'm sure it has a good number of inefficiencies and bad design decisions that the programmers using it had to work around. Yet despite it being a poor language in your opinion, they built successful programs with it. I wouldn't feel right calling them bad programmers. (large portions of Facebook and Flickr, for example, are written in PHP. They used to be written entirely in PHP before migrating the bottlenecks out to lower-level languages as they scaled up... as is common in most high-level languages) It also depends on what you mean by better programmer. Some languages value
Re: Object Relational Mappers are evil (a meditation)
Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Mon, 21 Dec 2009 11:44:29 -0500, J Kenneth King wrote: A programmer that lacks critical thinking is a bad programmer. The language they use has no bearing on such human facilities. That's nonsense, and I can demonstrate it by reference to a single programming language, namely Python. For many years, Python had no ternary if operator: result = x if condition else y Instead the accepted, idiomatic Python way of writing this was to use short-circuit booleans: result = condition and x or y However this idiom is buggy! If x is a false-value (say, 0) then result gets set to y no matter what the value of condition. This buggy idiom survived many years of Python development, missed by virtually everyone. Even coders of the calibre of Raymond Hettinger (who neither lacks critical thinking nor is a bad programmer) have been bitten by this: The construct can be error-prone. When an error occurs it can be invisible to the person who wrote it. I got bitten in published code that had survived testing and code review: ... http://mail.python.org/pipermail/python-dev/2005-September/056510.html This is a clear and obvious case where a language feature (in this case, the lack of a feature) encouraged an otherwise excellent coder to make an error. It was a very subtle error, which was not picked up by the author, the tests, or the coder reviewer(s). Had Python been different (either by including a ternary if statement, or by forcing and/or to return bools only) then this bug never would have occurred. Of course awful programmers will be awful programmers in any language, and excellent programmers will be excellent programmers in many languages. (I say many rather than any deliberately. There's a reason why nobody uses languages like Brainf*ck, Whitespace, Ook or Intercal for real work.) But most coders are neither awful nor excellent. The language DOES make a difference: the quality of a technician depends partly on the quality of his tools, and programmers are no different. If you don't believe me, imagine writing code in a language without functions or loops, so you have to use GOTO for everything. All very true. But did the lack of ternary encourage Raymond to become a bad programmer? That is what I believe the core of the argument is. Sure the misfeature was over-looked by Raymond, but it took him (and perhaps the help of others) to recognize it and fix it. That's because he's human and the language is inert. He is smart and obviously has the cognitive capabilities to recognize that the language has to change in order to be a better tool. It would be a different story if he just assumed that the misfeature was actually a feature and that it was a good thing. In such a case would Python the language be at fault or the people who write programs with it? Good tools make all the difference in the world, I'm not arguing that. Just that the tools don't use us; we use them. Programming in Python doesn't instantly make me a better programmer. It can certainly make me think of myself as a good programmer though... ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Lie Ryan lie.1...@gmail.com writes: On 12/17/2009 3:17 PM, J Kenneth King wrote: A language is a thing. It may have syntax and semantics that bias it towards the conventions and philosophies of its designers. But in the end, a language by itself would have a hard time convincing a human being to adopt bad practises. Perhaps someone should make a research whether if you teach a language to kids, where one group is taught the language filtered from bad words and another group is taught all the languages' bad words on purpose. Will one group have more behavioral problems compared to the other? I would be curious to know, but the test is likely impossible without trespassing on ethical boundaries. ;) I would hypothesize that you would not find an increase in behavioural problems. a) Without cultural context bad words have little meaning b) Behavioural issues can be attributed to several factors such as physiology, health, environment, etc. c) This has nothing to do with programming languages. A programmer that lacks critical thinking is a bad programmer. The language they use has no bearing on such human facilities. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Fri, 11 Dec 2009 19:20:21 -0500, Steve Holden wrote: Simon Forman wrote: [...] As far as the OP rant goes, my $0.02: bad programmers will write bad code in any language, with any tool or system or environment they're given. If you want to avoid bad code there's (apparently) no substitute for smrt programmers who are familiar with the tools they're using, not just the syntax but the underlying conceptual models as well. Hear, hear! That's all very well, but some languages and techniques encourage the programmer to write bad code. That's just BS. Bad code doesn't just write itself. Programmers write bad code. And ignorance is not an excuse. Just because a language allows a programmer to write sloppy code doesn't put the language at fault for the bad code programmers write with it. Any half-way decent programmer should be cognisant of when they're writing bad code and when they're writing good code. They should be able to admit that they don't know enough about a language to be writing programs for money in it. They should be able to see anti-patterns and areas of their code that should be re-factored or re-written. The real underlying problem is the human characteristic that allows us to let ourselves believe that we're better than everyone else or more simply, better than we really are. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
r0g aioe@technicalbloke.com writes: J Kenneth King wrote: Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Fri, 11 Dec 2009 19:20:21 -0500, Steve Holden wrote: snip Hear, hear! That's all very well, but some languages and techniques encourage the programmer to write bad code. That's just BS. Bad code doesn't just write itself. Programmers write bad code. And ignorance is not an excuse. Just because a language allows a programmer to write sloppy code doesn't put the language at fault for the bad code programmers write with it. Okay, as long as you realize the corollary of your argument is: It is impossible for a language to encourage programmers to write good code and promote good programming practices by design. I'm not sure that's entirely true either. I think python's one way to do something design philosophy goes some way toward that, as does Smalltalk's enforced message passing. I think PHP's superglobals and namespacing encourage bad practices (or used to back in the day), as do Basic's GOTO and Ecmascript's prototype overriding. I think your corollary is slightly misleading. It would be more apt to say, Just because a language allows a programmer to write good code doesn't mean that the language is responsible for the good code programmers write with it. It is the responsibility of the programmer to recognize the advantages and flaws of their tools. PHP doesn't encourage a programmer to be a bad programmer because it lacks name-spaces or because BASIC has GOTO statements. A bad programmer will be a bad programmer because they don't understand what makes these features distinct, useful, or damaging. The language doesn't encourage anything. It's just a medium like oil paints and canvas. A painting can be good or bad despite the medium it is constructed on. The skill of the painter is what matters. Surely a language CAN be said to encourage kludges and sloppiness if it allows a good way and a bad way and makes the bad way much easier to implement or understand for noobs. Roger. The programmer can be encouraged to use kludges and produce sloppy code. Whether by ignorance or inflated ego. Languages with more choice just give them more rope to hang themselves with. -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Neil Cerutti ne...@norwich.edu writes: On 2009-12-16, J Kenneth King ja...@agentultra.com wrote: The language doesn't encourage anything. It's just a medium like oil paints and canvas. A painting can be good or bad despite the medium it is constructed on. The skill of the painter is what matters. Technically, oil paints do encourage a certain kind of painting. They can be layered on top of old paint easily, and they dry slowly, allowing you to slowly build up a painting in layers, and create effects with texture. If you try doing thse things with watercolors, and you'll probably be discouraged. I think a programming language does encourage a certain kind of code. Good code in one language can be poor in another. It's a weak analogy on my part, but I think I do understand what you mean here. In regards to my original point, I think I just came up with a clearer way to express it: A language is a thing. It may have syntax and semantics that bias it towards the conventions and philosophies of its designers. But in the end, a language by itself would have a hard time convincing a human being to adopt bad practises. I believe it's the fault of the programmer who adopts those poor practises. Surely their acceptance of GOTO statements and prototype-overloading are signs of their own preferences and ignorance? It suggests to me that they learnt enough of one language to get by and stopped thinking critically as soon as they sat in front of their keyboard. Throw an idiot behind a Python interpreter and it won't teach them a damn thing unless they're capable of learning it on their own. No matter how well you manage to hard code your conventions into the language. Bad code is written by bad programmers, not bad programming languages. -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl to Python conversion
martin.sch...@gmail.com (Martin Schöön) writes: First off: I am new here and this is my first post after lurking for quite some time. Hi. Second off: I don't know much Python---yet. It's not a very big language. If you have experience programming in other languages, you can probably pick it up in a day or two. Problem: I have come across a small open source application that I find quite useful. It does have one major flaw though. Its output is in imperial units. Converting isn't a big deal for occasional use but if I start to use this stuff on a regular basis... So I down-loaded the source code and found this thing is written in Perl. Should I learn enough Perl to add the conversion? Probably but this may be a nice excuse to get my Python education going and if I do I might as well re-do the user interface. Well you can always call it from Python via subprocess (which basically wraps a shell and has fancy ways putting data in and extracting data out). If I do re-write this thing in Python I might need to learn both Perl and Python... You'll need to know one of them rather well and enough of the other to get by. It's probably easier to know more Perl than Python since Perl is a lot more expressive than Python (in the TMTOWTDI sense). Frankly I learned Perl before Python and find it rather easy to go between the two. YMMV. Hence, are there any Perl to Python converters? So far I have only found bridgekeeper which really is (was?) consultancy. Apart from that I only find people recommending a manual re-write. It depends where the two languages vary from one another. If the script your translating uses basic types or even simple classes and typical control structures and operations then translating from one to the other is a simple matter of copy-pasting the code and translating the syntax and small bits of grammar. However, in areas where there are high variations; you'll probably want to stay away from it. Perl has a lot of freedom to manipulate references and the programmer can modify the language to suit their needs. So just be careful of code that uses these features as they are difficult to translate into Python. Any thoughts/recommendations? Depends: - If you needed it done yesterday to get some work done, wrap the Perl script in a subprocess and buy yourself some time to think it over. - If your purpose is to learn Python, then start from scratch. Use the Perl as a guide if there are any maths or algorithms you are unsure about. - If you're just hacking around to learn stuff, learn a little of both. It will make you smarter if it doesn't confuse the heck out of you and make you quit before you finish. ;) TIA, /Martin HTH -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about 'remote objects'
Frank Millman fr...@chagford.com writes: Hi all I am writing a multi-user business/accounting application. It is getting rather complex and I am looking at how to, not exactly simplify it, but find a way to manage the complexity. I have realised that it is logically made up of a number of services - database service with connection to database workflow engine for business processes services manager to handle automated services, such as web services client manager to service logins from client workstations possibly others I have made a start by splitting some of these off into separate modules and running them in their own threads. I am concerned about scalability if they are all running on the same machine, so I am looking into how to enable these services to run on separate servers if required. Have you finished the application already? At my job we're still serving just over 1M+ web requests (a month), processing 15k+ uploads, and searching through over 5M+ database records a day. We're still running on 3 boxes. You can get a lot out of your machines before you have to think about the complex task of scaling/distributing. My first thought was to look into Pyro. It seems quite nice. One concern I had was that it creates a separate thread for each object made available by the server. My database server creates separate objects for each instance of a row read in from the database, and with multiple users running multiple applications, with each one opening multiple tables, this could run into hundreds, so I was not sure if that would work. It probably will work. Pyro is a very nice framework and one that I've built a few applications on. It has a lot of flexible design patterns available. Just look in the examples included with the distribution. Then I read that the multiprocessing module allows processes to be spread across multiple servers. The documentation is not as clear as Pyro's, but it looks as if it could do what I want. I assume it would use processes rather than threads to make multiple objects available, but I don't know if there is a practical limit. There is a theoretical limit to all of the resources on a machine. Threads don't live outside of that limit. They just have a speedier start-up time and are able to communicate with one another in a single process. It doesn't sound like that will buy you a whole lot in your application. You can spawn as many processes as you need. Then I thought that, instead of the database server exposing each object remotely, I could create one 'proxy' object on the server through which all clients would communicate, and it in turn would communicate with each instance locally. That felt more managable, but then I thought - why bother with remote objects at all? Why not just run a SocketServer on the database server, and design a mini-protocol to allow clients to make requests and receive results. This is a technology I am already comfortable with, as this is how I handle client workstation logins. If I did go this route, I could apply the same principle to all the services. Because unless you wrote your own database or are using some arcane relic, it should already have its own configurable socket interface? I don't have the experience to make an informed decision at this point, so I thought I would see if there is any consensus on the best way to go from here. Finish building the application. Do the benchmarks. Profile. Optimize. Find the clear boundaries of each component. Build an API along those boundaries. Add a network layer in front of the boundaries. Pyro is a good choice, twisted is also good. Roll your own if you think you can do better or it would fit your projects' needs. Is there any particular benefit in using remote objects as opposed to writing a SocketServer? Abstraction. Pyro is just an abstraction over an RPC mechanism. Nothing special about it. Twisted has libraries to do the same thing. Writing your own socket-level code can be messy if you don't do it right. Any advice will be much appreciated. Thanks Frank Millman Best of luck. -- http://mail.python.org/mailman/listinfo/python-list
Re: How decoupled are the Python frameworks?
shocks benmari...@googlemail.com writes: Hi I'm getting back into Python after a long break. I've been developing large enterprise apps solely with Adobe Flex (ActionScript) for the past couple years. During that time I've used a number of 'MVC' frameworks to glue the bits together - among them Cairngorm, a modified implementation of Cairngorm using the Presentation Model pattern, PureMVC, Mate (an IOC container but with an MVC implementation) and Parsley (IOC but you have to roll-you-own MVC). During that time I've been in large teams (30 Flex + 30 Java) to small teams (2 Flex + 1 Java). The motivation of these frameworks is the decouple your concerns, allowing your apps to be more scalable, easier to test, and supposedly easier to maintain. Some do the decoupling better job than others, but there is also the question of how decoupled is your code from the framework? It's all well and good having something clever working behind the scenes wiring and routing everything together, but I wonder where this leaves the code base if the framework, which was selected at the beginning of the project, is replaced with something else months or years later (i.e. the framework just doesn't scale as expected, the community involvement dies and it's no longer maintained properly, etc). I've seen it happen and I've been experienced the pain of detangling massive amounts of code which is full of framework specific imports, methods and boilerplate code. And then there's updating the unit tests! My question is how good are the current crop of Python frameworks? I've used Django twice in production and didn't like that much. The implementation is Django specific for starters. I've picked up Pylons and I'm trying that out. I'm not sure how well it fares? I do feel a bit uneasy about the code generation that some of the Python frameworks do. Pylons creates something like 20 files for a 'helloworld'. It does do some great things out of the box, but I wonder where that leaves your own code. After spending 3-6 months on your Pylons webapp, how easy is it to move to something else? Maybe one of the Python IOC once they mature. What are some good techniques people are using to future (framework) proof their apps? I'm interested to hear people experiences with the various frameworks and how decoupled their code is from them. The best of the current Flex frameworks for me is Parsley. The only noticeable Parlsey code is an '[Inject]' meta tag here and there and a couple import statements. All the complicated object creation and messaging is done higher up the chain. Cheers, Ben I've stayed away from the Java world at least professionally... but I think I understand where you're getting. I'm not a huge fan of web development. Which is rather counter-intuitive for me to say since it's been the bulk of my work for the past four years. It's because there's no easy way to get around the warts. Websites are one thing but to try and think of these things as applications becomes an expensive illusion to maintain. The problem with thinking about these things as applications with an interface, a controller, and a model is: statelessness! Oh also serialization. Getting around these issues are pretty much the raison d'etre for web frameworks. Without them we end up with PHP. As far as swappable Python web frameworks... NONE. AFAIK, they all require some commitment to tying your application to them. However, there are a crop of frameworks that do minimize the pain of such a decision: web.py and bobo are two that I've been working with (though it sounds like cherrypy would be very good at separating dispatching from application code). You could of course write your stuff closer to the metal so to speak... WSGI would be about as low as I go. -- http://mail.python.org/mailman/listinfo/python-list
Re: Go versus Brand X
a...@pythoncraft.com (Aahz) writes: Comparing Go to another computer language -- do you recognize it? http://www.cowlark.com/2009-11-15-go/ If you skip to the conclusion, you'll be better off. The author has an interesting point. Go (the language) is not really ground-breaking. I don't understand why they're so busy creating their own walled garden... Their own browser, their own programming languages, their own file systems, etc. Weird. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Programming Challenges for beginners?
astral orange 457r0...@gmail.com writes: Hi- I am reading the online tutorial along with a book I bought on Python. I would like to test out what I know so far by solving programming challenges. Similar to what O'Reilly Learning Perl has. I really enjoyed the challenges at the end of the chapter and it really help me test out if I was truly taking in the material like I should. Could I get some suggestions on resources? Is there anywhere where I can go online (for free or purchase) for programming problems? I am familiar with sites like Code Chef...etc...but at this stage that is not the right 'venue' for me. I mainly need challenges like the ones they have in Learning Perl. Thanks again for all the help, 457r0 http://www.pythonchallenge.com/ I find this one neat because it uses riddles rather than straight math. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python/HTML integration: phileas v0.3 released
papa hippo hippost...@gmail.com writes: On 20 nov, 09:02, Stefan Behnel stefan...@behnel.de wrote: papa hippo, 19.11.2009 19:53: The prime goal of 'phileas' is to enable html code to be seamlessly included in python code in a natural looking syntax, without resorting to templatng language. I assume you know XIST, ElementTree's ElementMaker, and all those other ways of generating XML/HTML from Python code in a natural looking way? Stefan Hi Stefan, Thanks for your feedback. Yes, I am aware that phileas might - on the basis of the short description on this post - come across like a 're-invented wheel'. There is, however, one big difference between phileas and all other other similar packages (XIST, ELementTree, HTMLgen, HyperText, pyhtmloo etc.) that I inspected: Phileas uses distinct objects to generate each start and end tag, whereas all the others use a single function call (in some cases itself generated by a function call) to generate a complete well- formed element including start-tag and (where required) end-tag. In theory this is less neat and indeed it means one can write 'bad' HTML (e.g. missing end of paragraphs) with phileas just as easily as when writing pure html. In practice, however, I find it at a lot easier to use. While using pyhtmloo (my previous favourite HTML generator), I had found myself using awkward complicated artificial constructions in order to generate all but the simplest HTML - and spent much time playing 'hunt the missing bracket'. With phileas, these complexities seem to just fall away. Any decent editor should be able to balance parenthesis for you. Put another way, Phileas generates HTML4.0 - warts and all; it is not a parser or generator of XML. I'm considering building in checks/warnings for unclosed elements etc., probably in the next-but-one pre-release. That your library will require a validation to be executed at run-time seems like it will be tedious to use. A decent text editor can even balance your HTML tags for you. Though you have a neat DSL like language for representing HTML elements. I'd suggest taking it one step further and creating a machine that can read in a Python data-structure and with as few hints as possible wrap it in the appropriate tags. Larry -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl conversion to python...
Benjamin Schollnick bscholln...@gmail.com writes:
Folks,
I'm having some issues here with pyserial trying to translate a perl
script to python... It's probably my inexperience with PySerial
perl that is troubling me...
Can anyone assist?
I'm concerned, since I can't seem to receive the data in any reliable
manner.. I've tested multiple times, and only once received data...
So I suspect that my Transmit receive code is faulty...
def xmit ( data, serialport ):
for x in data:
xmit_byte (x, serialport)
# serialport.write ( binascii.unhexlify ( data ) )
# for x in data:
# print str(x).encode ('hex')
# serialport.write ( x.encode('hex'))
def receive ( serialport ):
received = serialport.read (20)
print received, !
Gah.. indentation is broken in your post... :S
- Perl Code
sub tx_command {
my $port = shift;
my $cmd = shift;
# warn tx_command($cmd)\n;
my @cmd_bytes = split(/\s/, $cmd);
foreach my $byte (@cmd_bytes) {
$byte = pack('C', hex($byte));
$port - write($byte);
select(undef, undef, undef, 0.01);
}
}
import struct
def tx_command(port, cmd):
cmd_bytes = cmd.split(' ')
for byte in cmd_bytes:
byte = struct.pack('C', hex(int(byte)))
port.write(byte)
# select() is a system call in Perl..
# insert Python equivalent here
# returns the rtt, or 0 if no response
sub rx_response {
my ($port, $address) = @_;
$port-read_char_time(0); # don't wait for each character
$port-read_const_time(5000); # timeout if we don't get what we're
looking for
my $buf = '';
my $t_start = time;
### accumulate one byte at a time until we see the substring we're
looking for
while (1) {
my ($count_in, $string_in) = $port-read(1);
if ($count_in == 0) {
# warn TIMEOUT\n;
return 0;
}
$buf .= $string_in;
my $bufstring = packed_to_text($buf);
#warn bufstring: .$bufstring;
if ($bufstring =~/02 50 $address (.. .. ..) (..) (.. ..)/) {
my $powerlinc_addr = $1;
my $flags = $2;
my $command = $3;
# warn got response!\n;
my $rtt = time() - $t_start;
return $rtt;
}
}
}
This isn't all that more complicated. I dunno, maybe it's just me but
I find most Perl pretty easy to read (barring obfuscation which just
takes longer to read but is no more difficult). For the most part you
can generally substitute the Python equivalent statements for each
line of Perl and get good results. Optimize for better Pythonicity
afterwards.
--
http://mail.python.org/mailman/listinfo/python-list
Re: python simply not scaleable enough for google?
David Cournapeau courn...@gmail.com writes: On Tue, Nov 17, 2009 at 10:48 PM, Aaron Watters aaron.watt...@gmail.com wrote: I don't think Python and Go address the same set of programmer desires. For example, Go has a static type system. Some programmers find static type systems to be useless or undesirable. Others find them extremely helpful and want to use them them. If you're a programmer who wants a static type system, you'll probably prefer Go to Python, and vice versa. That has nothing to do with implementation speed or development expenditures. If Google spent a million dollars adding static types to Python, it wouldn't be Python any more. ... and I still have an issue with the whole Python is slow meme. The reason NASA doesn't build a faster Python is because Python *when augmented with FORTRAN libraries that have been tested and optimized for decades and are worth billions of dollars and don't need to be rewritten* is very fast. It is a bit odd to dismiss python is slow by saying that you can extend it with fortran. One of the most significant point of python IMO is its readability, even for people not familiar with it, and that's important when doing scientific work. Relying on a lot of compiled libraries goes against it. I think that python with its scientific extensions is a fantastic tool, but I would certainly not mind if it were ten times faster. In particular, the significant cost of function calls makes it quickly unusable for code which cannot be easily vectorized - we have to resort to using C, etc... to circumvent this ATM. Another point which has not been mentioned much, maybe because it is obvious: it seems that it is possible to makes high level languages quite fast, but doing so while keeping memory usage low is very difficult. Incidentally, the same tradeoff appears when working with vectorized code in numpy/scipy. I think this is the only interesting point in the whole conversation so far. It is possible for highly dynamic languages to be optimized, compiled, and run really fast. The recent versions of SBCL can compile Common Lisp into really fast and efficient binaries. And Lisp could be considered even more dynamic than Python (but that is debateable and I have very little evidence... so grain of salt on that statement). It's possible, it just hasn't been done yet. PyPy is getting there, but development is slow and they could probably use a hand. Instead of waiting on the sidelines for a company to back PyPy developemnt, the passionate Python programmers worth a salt that care about Python development should contribute at least a patch or two. The bigger problem though is probably attention span. A lot of developers today are more apt to simply try the next new language than to roll of their sleeves and think deeply enough to improve the tools they're already invested in. David -- http://mail.python.org/mailman/listinfo/python-list
Re: object serialization as python scripts
King animator...@gmail.com writes: Why is it easier than the above mentioned - they are *there* (except the custom xml), and just can be used. What don't they do you want to do? Other than that, and even security issues put aside, I don't see much difference between pickle and python code, except the latter being more verbose. Which suits humans, but other than that has no advantage. Diez My application is PyQt based and objects that I am trying to save are couple of QGraphicsItem instances. You can't save instances of QGraphicsItem using pickle or shelve. Custom xml format would be a real pain as you have to write code for writing/reading both. I am aware of security issue but over all there are only 5 statements I have to use to get the entire information back. I can do syntax checking and other stuff before I should execute the script. Another reason is that data should be in human readable form. Prashant Python 2.6.2 Win XP 32 Pickling should work just fine. If you cannot pickle the class with the default pickler, you can hook into the pickler protocol to tell pickle how to pickle instances of the class. Failing that you can write your own pickler class for handling the special case. Python scripts aren't really a good data format. They're not structured in a way that would be easy to parse and extract information from in a non-python context without a lot of work. -- http://mail.python.org/mailman/listinfo/python-list
Re: python simply not scaleable enough for google?
mcherm mch...@gmail.com writes: On Nov 11, 7:38 pm, Vincent Manis vma...@telus.net wrote: 1. The statement `Python is slow' doesn't make any sense to me. Python is a programming language; it is implementations that have speed or lack thereof. [...] 2. A skilled programmer could build an implementation that compiled Python code into Common Lisp or Scheme code, and then used a high-performance Common Lisp compiler... I think you have a fundamental misunderstanding of the reasons why Python is slow. Most of the slowness does NOT come from poor implementations: the CPython implementation is extremely well-optimized; the Jython and Iron Python implementations use best-in-the-world JIT runtimes. Most of the speed issues come from fundamental features of the LANGUAGE itself, mostly ways in which it is highly dynamic. In Python, a piece of code like this: len(x) needs to watch out for the following: * Perhaps x is a list OR * Perhaps x is a dict OR * Perhaps x is a user-defined type that declares a __len__ method OR * Perhaps a superclass of x declares __len__ OR * Perhaps we are running the built-in len() function OR * Perhaps there is a global variable 'len' which shadows the built-in OR * Perhaps there is a local variable 'len' which shadows the built-in OR * Perhaps someone has modified __builtins__ In Python it is possible for other code, outside your module to go in and modify or replace some methods from your module (a feature called monkey-patching which is SOMETIMES useful for certain kinds of testing). There are just so many things that can be dynamic (even if 99% of the time they are NOT dynamic) that there is very little that the compiler can assume. So whether you implement it in C, compile to CLR bytecode, or translate into Lisp, the computer is still going to have to to a whole bunch of lookups to make certain that there isn't some monkey business going on, rather than simply reading a single memory location that contains the length of the list. Brett Cannon's thesis is an example: he attempted desperate measures to perform some inferences that would allow performing these optimizations safely and, although a few of them could work in special cases, most of the hoped-for improvements were impossible because of the dynamic nature of the language. I have seen a number of attempts to address this, either by placing some restrictions on the dynamic nature of the code (but that would change the nature of the Python language) or by having some sort of a JIT optimize the common path where we don't monkey around. Unladen Swallow and PyPy are two such efforts that I find particularly promising. But it isn't NEARLY as simple as you make it out to be. -- Michael Chermside You might be right for the wrong reasons in a way. Python isn't slow because it's a dynamic language. All the lookups you're citing are highly optimized hash lookups. It executes really fast. The OP is talking about scale. Some people say Python is slow at a certain scale. I say that's about true for any language. Large amounts of IO is a tough problem. Where Python might get hit *as a language* is that the Python programmer has to drop into C to implement optimized data-structures for dealing with the kind of IO that would slow down the Python interpreter. That's why we have numpy, scipy, etc. The special cases it takes to solve problems with custom types wasn't special enough to alter the language. Scale is a special case believe it or not. As an implementation though, the sky really is the limit and Python is only getting started. Give it another 40 years and it'll probably realize that it's just another Lisp. ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: substituting list comprehensions for map()
Steven D'Aprano ste...@remove.this.cybersource.com.au writes: On Tue, 03 Nov 2009 10:22:28 -0500, J Kenneth King wrote: However in this case the procedure by which we derive the value is not important or even interesting. It is much more succinct to think of the operation as a value and express it accordingly. There's no need to clutter the mind with extra name bindings and iteration keywords. They won't make our idea any more clear. dot_product = map(mul, vec1, vec2) vs dot_product = [a * b for a, b in zip(vec1, vec2)] It's very clear, at least to me, what a dot-product is in this case. Except it's not. The dot product of two vectors returns a scalar, not another vector: http://en.wikipedia.org/wiki/Dot_product So what you want is: dot_product = sum(map(mul, vec1, vec2)) Derh. Thanks for the catch. My bad. Adding in the loop construct and name bindings doesn't enhance my understanding of what a dot-product is. I don't need to see the loop construct at all in this case. A dot product is simply the multiplication of each element in a vector sequence. What you need is to define a function dot-product, and not hijack the name for a local value. Then the function's implementation is irrelevant to you: it could use a list comp, or could use map, it could use a for- loop, a while loop, recursion, or black magic: scalar = dot_product(vec1, vec2) Even better. But now I'm afraid the example is running away from the point. So to summarize: 1. Extra name bindings and loop keywords aren't always easier to read. 2. Expressing what we want rather than how we get it is much more clear. and (third dirty argument added) 3. List comprehensions leak their name bindings to the surrounding scope. :p Have a nice day. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: substituting list comprehensions for map()
Steven D'Aprano ste...@remove.this.cybersource.com.au writes: On Sun, 01 Nov 2009 23:54:16 -0800, Jon P. wrote: I'd like to do: resultlist = operandlist1 + operandlist2 where for example operandlist1=[1,2,3,4,5] operandlist2=[5,4,3,2,1] and resultlist will become [6,6,6,6,6]. Using map(), I can do: map(lambda op1,op2: op1 + op2, operandlist1, operandlist2) If the two lists are very large, it would be faster to use this: from operator import add map(add, operandlist1, operandlist2) This is the best solution so far. Is there any reasonable way to do this via a list comprehension ? [x+y for (x, y) in zip(operandlist1, operandlist2)] If the lists are huge, you can save some temporary memory by replacing zip with itertools.izip. I understand the OP was asking for it, but list comprehensions aren't the best solution in this case... it would just be line noise. List comprehensions are good for one-off transformations where it would only create a one-time method for map to use. -- http://mail.python.org/mailman/listinfo/python-list
Re: What IDE has good git and python support?
Aweks a...@ewadev.com writes: what do you use? emacs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python + twisted = Raindrop (in part)
Terry Reedy tjre...@udel.edu writes: Rrom: First look: inside Mozilla's Raindrop messaging platform http://arstechnica.com/open-source/news/2009/10/first-look-inside-mozillas-raindrop-messaging-platform.ars The backend components that are responsible for retrieving and processing messages are coded in Python on top of the Twisted networking framework. Ok, front end is html/Javascript, DB is erlang-based tjr There are already developers grumbling about using Twisted. https://wiki.mozilla.org/Raindrop/BackEndRoadmap#untwist_me.3F I checked out the source myself and looked at the roadmaps and bug lists to see if there were any patches I could work on. But it doesn't look all that interesting unless you're into Javascript. Python is mainly used to fetch data. Scheduling the fetch is left up to the user/OS. Python is also used for some extensions which are basically document transformation scripts. The extensions are stored in the CouchDB and are eval'd asynchronously on each document in the CouchDB. A fairly expensive process to be sure. Anyway, it's still really early in development so I'm sure they could use some help. :) PS: Anyone wanna take a stab at their twisted pro/con list? Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: Object Relational Mappers are evil (a meditation)
Aaron Watters aaron.watt...@gmail.com writes: On Oct 16, 10:35 am, mario ruggier mario.rugg...@gmail.com wrote: On Oct 5, 4:25 pm, Aaron Watters aaron.watt...@gmail.com wrote: Occasionally I fantasize about making a non-trivial change to one of these programs, but I strongly resist going further than that because the ORM meatgrinder makes it somewhere between extremely unpleasant and impossible to make any non-trivial changes to a non-trivial program, especially after it has been populated with data. Object-relational mappers are like putting lipstick on a pig:http://gizmoweblog.blogspot.com/2006/10/putting-lipstick-on-pig.html m ;-) Cute, but wrong. Using ORMs is better than using Object databases. In my case I use Python to un data created by java/hibernate. If I was using a java based object database I would be simply stuck. At least if you use an ORM you have a way to access the information without writing a program in the programming language that the ORM was defined in. Anyway, thanks for all the great comments on this thread from all you Sarcopterygii and Haplorrhini out there. Data persistence isn't a one-size fits all problem. It really depends on the needs of the system. Object databases solve the problem of storing complex object graphs, deeply nested data structures, and serializing language-specific objects like continuations or what-have-you (but I think that last one is yet unsolved). We all know what RDBMS' are good for. Neither is perfect for solving every data persistence situation. -- http://mail.python.org/mailman/listinfo/python-list
Re: nested structure with internal references
Hendrik van Rooyen hend...@microcorp.co.za writes: On Friday, 25 September 2009 19:11:06 Torsten Mohr wrote: I'd like to use a nested structure in memory that consists of dict()s and list()s, list entries can be dict()s, other list()s, dict entries can be list()s or other dict()s. The lists and dicts can also contain int, float, string, ... This sounds terribly convoluted. What are you trying to do? But i'd also like to have something like a reference to another entry. Everything and its brother in Python is an object, and things like lists and dicts reference objects automagically. I'd like to refer to another entry and not copy that entry, i need to know later that this is a reference to another entry, i need to find also access that entry then. Depending on how I read this, it is either trivial or impossible: A name is bound to an object by assignment. An object can have many names bound to it. A name can, at different times, refer to different objects. An object does not know which names are bound to it, or in what sequence it was done. So you can go from name to object, but not the other way around. You can use the same name in a loop to refer to different objects, and in each iteration, use the name to store a reference to the current object in a list or dict. Is something like this possible in Python? Not too sure what you are trying to do. The references only need to refer to entries in this structure. The lists may change at runtime (entries removed / added), so storing the index may not help. You could start off with a list of lists of lists, to any level of nesting that you want. This will give you a structure like a tree with branches and twigs and leaves, but I am not sure if this is what you want or need, or if a flat structure like third normal form would suffice, or if you need a relational database. - Hendrik I'm not really sure what he wants either, but it sounds suspiciously like a linked list. In regards to the OP, though not a true linked-list, Python objects can be built with classes that describe essentially the same functionality. Just be careful about keeping references to large in-memory objects (see weakref) and try to avoid circular references in your mappings. Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: SQLite or files?
ici iltch...@gmail.com writes: I like shelve for saving small amounts of data, user preferences, recent files etc. http://docs.python.org/library/shelve.html I like it too, but I hear the great powers that be are going to deprecate it. For Qt use QtCore.QCoreApplication.setOrganizationName, QtCore.QCoreApplication.setApplicationName than setValue, value from QtCore.QSettings. -- http://mail.python.org/mailman/listinfo/python-list
escaping characters in filenames
I wrote a script to process some files using another program. One thing I noticed was that both os.listdir() and os.path.walk() will return unescaped file names (ie: My File With Spaces Stuff instead of My\ File\ With\ Spaces\ \\ Stuff). I haven't had much success finding a module or recipe that escapes file names and was wondering if anyone could point me in the right direction. As an aside, the script is using subprocess.call() with the shell=True parameter. There isn't really a reason for doing it this way (was just the fastest way to write it and get a prototype working). I was wondering if Popen objects were sensitive to unescaped names like the shell. I intend to refactor the function to use Popen objects at some point and thought perhaps escaping file names may not be entirely necessary. Cheers -- http://mail.python.org/mailman/listinfo/python-list
Re: escaping characters in filenames
Nobody nob...@nowhere.com writes: On Wed, 29 Jul 2009 09:29:55 -0400, J Kenneth King wrote: I wrote a script to process some files using another program. One thing I noticed was that both os.listdir() and os.path.walk() will return unescaped file names (ie: My File With Spaces Stuff instead of My\ File\ With\ Spaces\ \\ Stuff). I haven't had much success finding a module or recipe that escapes file names and was wondering if anyone could point me in the right direction. As an aside, the script is using subprocess.call() with the shell=True parameter. There isn't really a reason for doing it this way (was just the fastest way to write it and get a prototype working). I was wondering if Popen objects were sensitive to unescaped names like the shell. I intend to refactor the function to use Popen objects at some point and thought perhaps escaping file names may not be entirely necessary. Note that subprocess.call() is nothing more than: def call(*popenargs, **kwargs): return Popen(*popenargs, **kwargs).wait() plus a docstring. It accepts exactly the same arguments as Popen(), with the same semantics. If you want to run a command given a program and arguments, you should pass the command and arguments as a list, rather than trying to construct a string. On Windows the value of shell= is unrelated to whether the command is a list or a string; a list is always converted to string using the list2cmdline() function. Using shell=True simply prepends cmd.exe /c to the string (this allows you to omit the .exe/.bat/etc extension for extensions which are in %PATHEXT%). On Unix, a string is first converted to a single-element list, so if you use a string with shell=False, it will be treated as the name of an executable to be run without arguments, even if contains spaces, shell metacharacters etc. The most portable approach seems to be to always pass the command as a list, and to set shell=True on Windows and shell=False on Unix. The only reason to pass a command as a string is if you're getting a string from the user and you want it to be interpreted using the platform's standard shell (i.e. cmd.exe or /bin/sh). If you want it to be interpreted the same way regardless of platform, parse it into a list using shlex.split(). I understand; I think I was headed towards subprocess.Popen() either way. It seems to handle the problem I posted about. And I got to learn a little something on the way. Thanks! Only now there's a new problem in that the output of the program is different if I run it from Popen than if I run it from the command line. The program in question is 'pdftotext'. More investigation to ensue. Thanks again for the helpful post. -- http://mail.python.org/mailman/listinfo/python-list
Re: Script runs manually, but cron fails
Bryan bryanv...@gmail.com writes: I have a backup script that runs fine when I run it manually from the command line. When I run it with cron, the script stops running at random points in the source code. The script calls rsync with the subprocess module, which in turn uses ssh to backup files from a box on my lan. It also uses the atexit module to always run a certain piece of cleanup code when the script exits. However, this piece of code is never called when I use cron to run the script, but a ps -A command also does not show the python process so I know the script is dead. The log files generated by the script all show rsync as completing, but then the logging gets cutoff at random places when the script dies after that point. I am not getting any email from cron complaining of an error either. The script runs fine in my bash shell, what could cron be doing to interfere? Double check nothing is writing to stdout/stdin Some cron's don't mind, but I always squelch output because some do. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ann: Google releases Python-based open-source NX server
Robert Kern robert.k...@gmail.com writes: On 2009-07-16 09:51, J Kenneth King wrote: jknjkn...@nicorp.f9.co.uk writes: Google quietly releases open-source NX server ...written in Python, apparently http://www.computerworld.com/s/article/9135504/ Google_quietly_releases_open_source_NX_server?taxonomyId=88 Neatx can be downloaded from Google's code repository: http://code.google.com/p/neatx/ Regards J^n It's pretty cool, but not PEP8! Probably because they just bought the source off of another smaller proprietary project. No, it' just that Google does not use PEP8 for its internal style standard. Yeah probably. I'm just a little OCD sometimes. -- http://mail.python.org/mailman/listinfo/python-list
Re: [0, 0, 0, 1, 1, 1, 0] ... remove all 0 values
Friðrik Már Jónsson frid...@pyth.net writes: ma wrote: filter(lambda x: x, your_list) Good call! Equivalent but more efficient: filter(None, your_list) Regards, Friðrik Már I was wondering when someone would mention filter() -- http://mail.python.org/mailman/listinfo/python-list
Re: Python/pyobjC Apps on iPhone now a possibility?
Dr Mephesto dnh...@googlemail.com writes: I have been following the discussion about python and pyobjc on the iphone, and it seemed to me that the app-store rules prohibited embedded interpreters; so, python apps are a no-no. But now it seems that the Rubyists have the option that we don't. It seems there is a company, http://rhomobile.com/home, that has an SDK that allows ruby programs to be embedded together with an interpreter in an app! More interesting is the fact that several of these hybrid apps seem to have been accepted on the itunes app store. Here's a quote from a representative, found on this blog: http://www.rubyinside.com/rhodes-develop-full-iphone-rim-and-symbian-apps-using-ruby-1475.html ...First of all, to clarify, we precompile all framework and app code down to Ruby 1.9 VM bytecode. This yields great performance advantages. We also disable eval and other dynamic execution aspects of Ruby. In the end, on all platforms your app gets compiled with our framework all into one single executable, indistinguishable from any other executable. But even if we were shipping a fullon Ruby interpreter without compiling to bytecode and leaving dynamic evaluation enabled (as has been well remarked in the blogosphere by now) App Store rule 3.3.2 does not disallow interpreters but only downloading code to be executed by the interpreter. So, the question is, can the same thing be done for Python apps? I love Python and all, but it'd be apt to ask, what's the point? The iPhone is running on what? A 400Mhz ARM processor? Resources on the device are already limited; running your program on top of an embedded Python interpreter would only be adding pressure to the constraints; even if it was an optimized interpreter. Might as well just suck it up and learn C/Objective-C .. it's really not that hard. It took me about a day to pick up the language and another two or three to finagle with the libraries. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python/pyobjC Apps on iPhone now a possibility?
Dr Mephesto dnh...@googlemail.com writes: Sure, I am learning Objective C already, but the syntax is really unfriendly after python. I think it really depends on the type of app you want to write. Anything held back by network delays or that sits around waiting for user input are perfectly acceptable target apps. If you need speed for something really intensive, falling back to C is still much nicer than coding in Objective C. I agree that having a certain basic understanding of objective C is a must, but having the option to use a coder-friendly language like Ruby or Python can cut development time dramatically. If Ruby can do it (and it generally slower than python), why can Python also get a legal iPhone interpreter? It can if you want to write the interpreter. I just don't see the point. I can understand wanting a higher level language than assembler or maybe even C, but that's precisely what Objective-C is. Unfortunately, while the hardware for these mobile platforms is getting better these days, it's still not where it needs to be to run programs written in interpreted languages, IMO. Users typically only interact with an app for around two minutes at a time. Your app needs to be as fast as it can be. It's one of the few areas of software development today where optimizations can be justified (the other are games and scientific computing). Trust me, if there were an SMS app for the iPhone that loaded faster than the one that ships with it, guaranteed it would sell like hot-cakes (if Apple would let it in the store of course). If you do write the interpreter, let me know. I would certainly experiment with it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python/pyobjC Apps on iPhone now a possibility?
Stef Mientki stef.mien...@gmail.com writes: So, the question is, can the same thing be done for Python apps? I love Python and all, but it'd be apt to ask, what's the point? The iPhone is running on what? A 400Mhz ARM processor? Resources on the device are already limited; running your program on top of an embedded Python interpreter would only be adding pressure to the constraints; even if it was an optimized interpreter. I don't know iPhone, but I've done some experiments with 400 MHz arm, running Windows Mobile, and found PocketPyGUI running very very well on these devices. cheers, Stef Mientki Sure, but it's pretty relative in the sense that it might be fast enough if I'm sitting around but too slow if I want to enter some information in the app before the next train comes. As a programmer, I don't really see the benefit of using an embedded interpreter on the iPhone. Objective-C isn't the greatest language, but it's easy to learn and well supported. It also compiles into some pretty speedy executables. If you can sacrifice a little run-time speed for your users in exchange for ease of development on your part, all the more to you. My original point was that I don't see the benefit in that decision. -- http://mail.python.org/mailman/listinfo/python-list
Re: Code that ought to run fast, but can't due to Python limitations.
a...@pythoncraft.com (Aahz) writes: In article mailman.2639.1246802753.8015.python-l...@python.org, Hendrik van Rooyen m...@microcorp.co.za wrote: But wait - maybe if he passes an iterator around - the equivalent of for char in input_stream... Still no good though, unless the next call to the iterator is faster than an ordinary python call. Calls to iterators created by generators are indeed faster than an ordinary Python call, because the stack frame is already mostly set up. I think Beazely demonstrated this in his talk on using the python 2.5 co-routines to setup an xml parser. I believe he benchmarked it roughly and the initial results were rather impressive. http://www.dabeaz.com/coroutines/ -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing: pool with blocking queue
masher vertesp...@gmail.com writes: My questions, then, is: Is there a more elegant/pythonic way of doing what I am trying to do with the current Pool class? Forgive me, I may not fully understand what you are trying to do here (I've never really used multiprocessing all that much)... But couldn't you just assign your own Queue object to the Pool instance? -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing: pool with blocking queue
masher vertesp...@gmail.com writes: On Jul 2, 12:06 pm, J Kenneth King ja...@agentultra.com wrote: masher vertesp...@gmail.com writes: My questions, then, is: Is there a more elegant/pythonic way of doing what I am trying to do with the current Pool class? Forgive me, I may not fully understand what you are trying to do here (I've never really used multiprocessing all that much)... But couldn't you just assign your own Queue object to the Pool instance? That's basically my question. It does not appear as though there is any straightforward way of doing this because of the design of Pool's __init__ method, which passes _taskqueue to several functions. Hence, even if I were to reassign _taskqueue after __init__, that wouldn't change anything. I think I understand. There are ways to modify the class before instantiating it, but even the most clever or elegant solution will still smell funny. I suppose this might be worth submitting as a feature suggestion to the multiprocessing project. Best of luck. -- http://mail.python.org/mailman/listinfo/python-list
Re: fastest native python database?
per perfr...@gmail.com writes: hi all, i'm looking for a native python package to run a very simple data base. i was originally using cpickle with dictionaries for my problem, but i was making dictionaries out of very large text files (around 1000MB in size) and pickling was simply too slow. i am not looking for fancy SQL operations, just very simple data base operations (doesn't have to be SQL style) and my preference is for a module that just needs python and doesn't require me to run a separate data base like Sybase or MySQL. does anyone have any recommendations? the only candidates i've seen are snaklesql and buzhug... any thoughts/benchmarks on these? any info on this would be greatly appreciated. thank you berkeley db is pretty fast. locking and such nice features are included. the module you'd be looking at is bsddb i believe. -- http://mail.python.org/mailman/listinfo/python-list
Re: what I would like python.el to do (and maybe it does)
Piet van Oostrum p...@cs.uu.nl writes: J Kenneth King ja...@agentultra.com (JKK) wrote: JKK I find that it does work, but unlike SLIME for lisp, it just imports the statement. JKK It confused me at first, but basically the interpreter doesn't provide JKK any feedback to emacs. JKK Try opening a python source file (start python-mode if you don't have JKK an autoload hook) and do C-c C-z to bring up the Python JKK interpreter. Type in a simple assignment statement (like a = 1 + 2 JKK without the quotes) into the source file. Then just C-c C-c as JKK usual. I never get any feedback. Just C-x o to the interpreter and JKK print out the variable you just defined. It should be there. What kind of feedback do you expect? Well, that's the thing -- type a statement into a python interpreter and you just get a new prompt. LISP has a REPL, so you get some sort of feedback printed. However, some sort of visual cue on the emacs side would be nice. Either just flash the block of code being sent or a minibuffer message would be nice. Look for some SLIME tutorial videos on youtube to see some great interpreter - editor interaction. The stock Python interpreter probably wouldn't cut it close to something like SLIME in terms of features, but the iPython package might be a start. -- http://mail.python.org/mailman/listinfo/python-list
Re: What text editor is everyone using for Python
Lacrima lacrima.ma...@gmail.com writes: I am new to python. And now I am using trial version of Wing IDE. But nobody mentioned it as a favourite editor. So should I buy it when trial is expired or there are better choices? That is a slightly better question. Try some of the free alternatives. I do happen to use emacs. It took me quite a lot of adjusting to get used to it after being a vim user for almost ten years. Doesn't cost anything to give one a shot and see if it works for you. The choice of editor is a personal one (hence the reluctance to answer your original question). -- http://mail.python.org/mailman/listinfo/python-list
Re: which database is suitable for small applications
Jean-Michel Pichavant jeanmic...@sequans.com writes: Kalyan Chakravarthy wrote: Hi All, can any one suggest me which database I can use for my small application(to store user names ,passwords, very few other data.. ) I am using Python, Google Apps and guide me how to connect to database, I am very new to these technologies Please help me Thanks in advance -- Regards Kalyan If you are really new to these technologies and don't want to spend some time on it, just serialize on the disk your data structure containing your names, password and so on (http://docs.python.org/library/pickle.html). Depending on your application you may not be that concerned with security/reliability issues. Of course if you want to learn more about it, go on. I personally use postgreSQL with the pgdb python module. But this is the only one I ever used so I'll let someone else more familiar with all the database types respond to you. Jean-Michel sqlite is also a pretty lite database. Sits in a single file and the libraries ship with Python ( 2.6 I think? or maybe 2.5?). Serialization is always a neat option. With a hash table DB like berkeley or some such you can get a pretty speedy (albeit simple) persistence layer. Really depends on what your app needs. -- http://mail.python.org/mailman/listinfo/python-list
Re: what I would like python.el to do (and maybe it does)
Giovanni Gherdovich giovanni.gherdov...@sophia.inria.fr writes: Hello everybody, basically I'm writing here since I cannot make my python.el work (a major mode for writing python with emacs), but I would also like to share my user experience and tell you what I think an emacs mode should do, why do I like them and hopefully have some feedbacks to see if I misunderstood/underestimate something. == 1) my python.el doesn't behave very well == I learnt somewhere http://www.emacswiki.org/emacs/PythonMode that there are two major emacs mode for python around: python-mode.el and python.el. Asking my Emacs 22.3.1 about the variable load-path issuing 'C-h v load-path RET I see that /usr/share/emacs/22.3/lisp/progmodes is in that path; there I find a file named python.elc which I assume to be some kind of emacs lisp bytecode since is pretty much unreadable. So I searched the web for a plain version of it, finding that the feature I use more, i.e. sending a line of a text file to the python buffer for evaluation (see below), correspond to the key sequence \C-c\C-c (well, it's python-send-buffer, so maybe not a single line but the whole buffer; the closest to my needs, anyway). However: I open my Emacs, issue M-x python-mode, then M-x run-python to have the interpreter in a second buffer, I type something in the first buffer and then C-c C-c, but nothing happens. Am I missing something? Do I have any hope of having some sort of send-line-to-python-buffer function working? == 2) How do I use emacs modes for interpreted languages == Please note that what follows is just the personal perspective of an unexperienced user. Syntax highlighting is a great thing, but is not as critical to me as the feature I describe below. When I work with interpreted languages, I really hate doing it in the shell; after 20 commands I easily lose control on what happens and on which definitions are around. I use Emacs instead, so that I can have two buffers; in the first I type my expressions, in the second I evaluate them using some key bindings so that I can easily send the text from the first buffer to the second one line by line. In this way I can easily refactor my code, and eventually package it in a script if I like. Usually after a while the intepreter buffer is a big mess, so I restart it but my code is safe and sound in the first buffer. To do so, I don't really need a major mode, I admit; I just need to put the following code in my .emacs: (fset 'send-line-other-window [?\C-a ?\C- ?\C-e ?M-w right ?C-x ?o ?C-y return ?\C-x ?o]) (global-set-key [f11] 'send-line-other-window) Then I open emacs, C-x 2 to have a second buffer, C-x o to switch to it and M-x shell to run bash in it. Then, in the case of python, I run python in the bash buffer. Then I type my code in the first and with F11 I send lines to the interpreter. But since i like to do it The Right Way, I would like to let the python-mode worry about this... Sorry if this is just a bunch of obvious thoughts to most of you. Regards, Giovanni I find that it does work, but unlike SLIME for lisp, it just imports the statement. It confused me at first, but basically the interpreter doesn't provide any feedback to emacs. Try opening a python source file (start python-mode if you don't have an autoload hook) and do C-c C-z to bring up the Python interpreter. Type in a simple assignment statement (like a = 1 + 2 without the quotes) into the source file. Then just C-c C-c as usual. I never get any feedback. Just C-x o to the interpreter and print out the variable you just defined. It should be there. -- http://mail.python.org/mailman/listinfo/python-list
Re: What text editor is everyone using for Python
LittleGrasshopper seattleha...@yahoo.com writes: With so many choices, I was wondering what editor is the one you prefer when coding Python, and why. I normally use vi, and just got into Python, so I am looking for suitable syntax files for it, and extra utilities. I dabbled with emacs at some point, but couldn't get through the key bindings for commands. I've never tried emacs with vi keybindings (I forgot the name of it) but I've been tempted. So what do you guys use, and why? Hopefully we can keep this civil. Google should provide you with millions of responses. -- http://mail.python.org/mailman/listinfo/python-list
Re: Annoying feedparser issues
John Nagle na...@animats.com writes: This really isn't the fault of the feedparser module, but it's worth mentioning. I have an application which needs to read each new item from a feed as it shows up, as efficiently as possible, because it's monitoring multiple feeds. I want exactly one copy of each item as it comes in. In theory, this is easy. Each time the feed is polled, pass in the timestamp and ID from the previous poll, and if nothing has changed, a 304 status should come back. Results are spotty. It mostly works for Reuters. It doesn't work for Twitter at all; Twitter updates the timestamp even when nothing changes. So items are routinely re-read. (That has to be costing Twitter a huge amount of bandwidth from useless polls.) Some sites have changing feed etags because they're using multiple servers and a load balancer. These can be recognized because the same etags will show up again after a change. Items can supposedly be unduplicated by using the etag value. This almost works, but it's tricker than one might think. On some feeds, an item might go away, yet come back in a later feed. This happens with news feeds from major news sources, because they have priorities that don't show up in RSS. High priority stories might push a low priority story off the feed, but it may come back later. Also, every night at 00:00, some feeds like Reuters re-number everything. The only thing that works reliably is comparing the story text. John Nagle I can't really offer much help, but I feel your pain. I had to write something of a similar system for a large company once and it hurt. They mixed different formats with different protocols and man it was something in the end. The law of fuzzy inputs makes this stuff tough. It may help to create a hash from the first x number of bytes of the article text. Then cache all the hashes in a local dbm-style database. We used berkely, but it doesn't really matter. Whatever way you can generate and store a keyed signature will allow you to do a quick look up and see if you've already processed that article. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Thu, 07 May 2009 13:28:10 -0400, J Kenneth King wrote: Steven D'Aprano ste...@remove.this.cybersource.com.au writes: On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote: Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list- comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. They are tricky and need to be used carefully, *if at all*. IMO this means that if there's a way to do it without a nested list comprehension, then that solution should be preferred. Earlier, you claimed that list comps in general were discouraged: List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Ooops. Typo. My bad. Had it been quoted earlier it would have saved a few posts for sure. Such is the Internet. :) Emile said I'd be surprised to find routine usage frowned upon, giving you the opportunity to correct his (mis)understanding. You failed to do so, which I took as meaning that you agreed that routine usage of simple list comps were frowned upon. Now you're talking about *nested* list comps. You started off (apparently) objecting to list comps in general, because they can make readers apprehensive. Now you seem to be saying that it's only the complicated, overly-dense ones which rightly make readers apprehensive which you object to. I was rather confused by that. I use list comps all the time and said several times that I don't object to list comps, just nested ones. I suppose we're making progress if we agree that the Python docs warn against unnecessarily complicated nested list comps. Whether it discourages as well as warns is a matter of interpretation. But there's certainly no sign that list comps as a general technique are discouraged just because overly-complicated list comps are tricky to read. The same can be said about *any* piece of code. Well I did my best to make my interpretation clear. If the documentation says that nested list comps are difficult to read and should be used rarely, if at all, then I generally consider that discouraging their use. Meaning of course that in the right situation one may be able to justify their own use of a nested comp. However, for every day problems one wouldn't be encouraged to use them so liberally as some tend to do. So.. we have some sort of consensus then? This might be a rare phenomenon on Usenet... :) Cheers, J. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Terry Reedy tjre...@udel.edu writes: J Kenneth King wrote: Keep in mind that nested comprehensions are still available because they do have a use case that justifies their existence. Nested comprehensions are available because because the syntax makes them available by default and making a fiddly exception would be contrary to Python's style. A list comp creates an iterable. A list comp requires use of an iterable. Therefore, a list comp may use a list comp. However, I think the Python documentation warns against their use because people might rely on them for problems where they aren't necessary and since they are difficult to read... it can lead to difficult to read code. Whenever the expression that results in the iterable used by a list comp is sufficiently complex, readability is improved by pulling it out as a separate statement. Nested list comps are often examplex of such sufficiently complex expressions, but not the only possible one. tjr Agreed. A very succinct explanation of the point I was trying to make. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Steven D'Aprano ste...@remove.this.cybersource.com.au writes: On Wed, 06 May 2009 09:48:51 -0400, J Kenneth King wrote: Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list- comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. How does this discourage the use of list comprehensions? At most, it warns that complicated list comps are tricky. Complicated *anything* are tricky. They are tricky and need to be used carefully, *if at all*. IMO this means that if there's a way to do it without a nested list comprehension, then that solution should be preferred. and In real world, you should prefer builtin functions to complex flow statements. That's ridiculous. The example given is a special case. That's like saying Loops are hard, so in the real world, if you want a loop, find a builtin function that does what you want instead. What's the builtin function we're supposed to prefer over a complex flow statement like this? It's not ridiculous and says nothing of the sort. You're jumping to conclusions. If you have a problem with it, I suggest you complain to the Python documentation people. I don't make this stuff up. Just look at many of the example solutions provided in this thread to the OP's problem. The examples in the link I provided are also good as are ones provided in the itertools documentation. Keep in mind that nested comprehensions are still available because they do have a use case that justifies their existence. However, I think the Python documentation warns against their use because people might rely on them for problems where they aren't necessary and since they are difficult to read... it can lead to difficult to read code. -- http://mail.python.org/mailman/listinfo/python-list
Re: Best practice for operations on streams of text
James rent.lupin.r...@gmail.com writes: Hello all, I'm working on some NLP code - what I'm doing is passing a large number of tokens through a number of filtering / processing steps. The filters take a token as input, and may or may not yield a token as a result. For example, I might have filters which lowercases the input, filter out boring words and filter out duplicates chained together. I originally had code like this: for t0 in token_stream: for t1 in lowercase_token(t0): for t2 in remove_boring(t1): for t3 in remove_dupes(t2): yield t3 Apart from being ugly as sin, I only get one token out as StopIteration is raised before the whole token stream is consumed. Any suggestions on an elegant way to chain together a bunch of generators, with processing steps in between? Thanks, James Co-routines my friends. Google will help you greatly in discovering this processing wonder. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Emile van Sebille em...@fenx.com writes: On 5/5/2009 9:15 AM J Kenneth King said... List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. Can you provide a link for this? I'd like to see specifically what's being discouraged, as I'd be surprised to find routine usage frowned upon. Emile http://docs.python.org/tutorial/datastructures.html#nested-list-comprehensions If you’ve got the stomach for it, list comprehensions can be nested. They are a powerful tool but – like all powerful tools – they need to be used carefully, if at all. and In real world, you should prefer builtin functions to complex flow statements. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Emile van Sebille em...@fenx.com writes: On 5/1/2009 7:31 AM J Kenneth King said... Chris Rebert c...@rebertia.com writes: b = [] for pair in a: for item in pair: b.append(item) This is much more clear than a nested comprehension. I love comprehensions, but abusing them can lead to really dense and difficult to read code. I disagree on dense and difficult, although I'll leave open the question of abuse. Dense and difficult may be subjective to people like you or I, but you left out the to read part of that sentence. I was referring to the negative effect nested or complex list comprehensions can have on readability. b = [ item for pair in a for item in pair ] It's a clever statement, but use it once and its gone. Why not use the expanded form in a function definition and get an even shorter, but clearer statement? b = flatten(a) Boom, done. This is exactly the code above expressed in comprehension form. It's worth knowing that a list comprehension is structured identically to the equivalent for loop. So it really is neither more dense nor more difficult to read. Further, you can tell immediately from the start of the list comprehension what you've got -- in this case a list of item(s). Here with some slight changes... a = [(1, 2), (3, 4, 7), (5, 6)] [ item for j in a if len(j)==2 for item in j if item % 2 ] [1, 5] ...opposed to... for j in a: ... if len(j)==2: ... for item in j: ... if item % 2: ... b.append(item) ... b [1, 5] Thanks for the lesson in list comprehensions, but I'm well aware of how they work. List comprehensions can make a reader of your code apprehensive because it can read like a run-on sentence and thus be difficult to parse. The Python documentation discourages their use and I believe for good reason. It's much easier to explain complex processes with a function rather than a single nested statement. YMMV, Emile It will apparently vary greatly. Depending on how much coffee I've had. ;) J -- http://mail.python.org/mailman/listinfo/python-list
Re: Any idea to emulate tail -f
Iain King iaink...@gmail.com writes: On May 5, 7:00 am, Joel Juvenal Rivera Rivera joel...@gmail.com wrote: I want to make something very similar to the command tail -f (follow a file), i have been trying with some while True and some microsleeps (about .1 s); did someone has already done something like this? And about the file is the apache acceslog of a site with a lot of traffic. Regards joe / Joel Rivera This is very interesting, about using Generator Expressions: http://209.85.229.132/search?q=cache:ZHrV4E0eTI8J:www.dabeaz.com/generators/Generators.pdf Relevant stuff about 'tail -f' is on page 39, but I'd read the whole thing if you can. Iain +1 I second this suggestion. I've used this method in many scripts of late and it is quite handy. He even takes it a little further and shows you some neat things you can do with it later on. -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension question
Chris Rebert c...@rebertia.com writes: On Thu, Apr 30, 2009 at 5:56 PM, Ross ross.j...@gmail.com wrote: If I have a list of tuples a = [(1,2), (3,4), (5,6)], and I want to return a new list of each individual element in these tuples, I can do it with a nested for loop but when I try to do it using the list comprehension b = [j for j in i for i in a], my output is b = [5,5,5,6,6,6] instead of the correct b = [1,2,3,4,5,6]. What am I doing wrong? Your comprehension is the identity comprehension (i.e. it effectively just copies the list as-is). What you're trying to do is difficult if not impossible to do as a comprehension. Here's another approach: b = list(itertools.chain.from_iterable(a)) And without using a library function: b = [] for pair in a: for item in pair: b.append(item) This is much more clear than a nested comprehension. I love comprehensions, but abusing them can lead to really dense and difficult to read code. -- http://mail.python.org/mailman/listinfo/python-list
Re: python list handling and Lisp list handling
Steven D'Aprano st...@remove-this-cybersource.com.au writes: On Sat, 25 Apr 2009 23:51:18 -0700, namekuseijin wrote: On Apr 26, 1:31 am, Steven D'Aprano st...@remove-this- cybersource.com.au wrote: On Fri, 24 Apr 2009 21:01:10 -0700, Carl Banks wrote: That's because Python lists aren't lists. Surely you meant to say that Lisp lists aren't lists? It-all-depends-on-how-you-define-lists-ly y'rs, Yeah, the List Processing language got it all wrong by not going with arrays like Python... Well, Lisp was invented in 1958, before anyone knew how to program *wink*. And 50+ years of development hasn't taught them anything. :p Guess you don't know anything about programming unless you're new... Seriously though, linked lists are not the only sort of list. That was my point: first define what is a list, and then we can debate what is or isn't a list. Even within linked lists, there are various different types, all with their own strengths and weaknesses: singly-linked lists, doubly-linked lists, circular lists, open lists, xor-lists, lists with or without sentinels, lists with internal and external storage, unrolled linked lists, and combinations of all of the above. And any sufficiently powerful language would allow the programmer to adapt to any type they needed. ;) Interesting topic though. -- http://mail.python.org/mailman/listinfo/python-list
Re: unpythonic use of property()?
Luis Zarrabeitia ky...@uh.cu writes: On Monday 20 April 2009 11:29:19 am J Kenneth King wrote: Changing the ID value would break things on the server, so I wanted to write the interface class to respect those conventions. Then, take this opportunity fix the server and prevent it from breaking once you change the ID, because: Unfortunately it's not my server to fix. I can suggest a patch, but that's it. I'm well aware that if a developer really wanted to, they could get around it no matter what I did, but I figure that if I at least make it really difficult it will be obvious that they're really going into dangerous territory. you will have to do it anyway. I think it's great, for you, that the language you are using makes it so extremely easy to bypass almost any access restriction that you may put in the data sent to your clients. That means that you won't be overconfident, that you are forced to make sound decisions from the beginning, and that you won't be trusting that your clients will be well behaved (even with very strong enforcement of data hiding, a malicious client could forge the connection). Then, when the server is safe from user data, by all means go and make sure that the clients (and even the server!) will complain as loudly as possible if they want to break encapsulation. Well, really all I'm doing is writing a light framework around xmlrpclib. The design allows me to keep this kind of exception highly localized, so its not really a huge practical issue. The problem I had was that I found two approaches to the same problem that appeared to be pedantic in difference. However, you are right of course -- logically the server should protect against this vulnerability itself instead of the implementation using my framework. -- http://mail.python.org/mailman/listinfo/python-list
Re: unpythonic use of property()?
Luis Zarrabeitia ky...@uh.cu writes: On Wednesday 22 April 2009 01:44:38 pm J Kenneth King wrote: Then, take this opportunity fix the server and prevent it from breaking once you change the ID, because: Unfortunately it's not my server to fix. I can suggest a patch, but that's it. Yes, that's unfortunate. Btw, when I re-read my phrase by itself, it seemed hostile... My apologies. I'm still not very good at expressing my thoughts in english. It's okay. You have great English. ;) Then, I guess, you have little choice. Mangle the name, hope that the server will get fixed. -- Luis Zarrabeitia (aka Kyrie) Fac. de Matemática y Computación, UH. http://profesores.matcom.uh.cu/~kyrie -- http://mail.python.org/mailman/listinfo/python-list
Re: unpythonic use of property()?
Carl Banks pavlovevide...@gmail.com writes: On Apr 17, 4:00 pm, Scott David Daniels scott.dani...@acm.org wrote: Carl Banks wrote: On Apr 17, 10:21 am, J Kenneth King ja...@agentultra.com wrote: Consider: code: class MyInterface(object): def __get_id(self): return self.__id id = property(fget=__get_id) def __init__(self, id, foo): self.__id = id self.foo = foo class MyInterface2(object): def __init__(self, id, foo): self._id = id self.foo = foo @property def id(self): return self._id ... I was recently informed that it was 'unpythonic' and have since been a little confused by the term. I've heard it bandied about before but never paid much attention. What is 'unpythonic'? What about this example is unpythonic? There are different reasons someone might have said it. ... Some people think attribute name-mangling is unpythonic. It's true that people sometimes mistakenly treat it a solid information hiding mechanism, but I wouldn't call its usage unpythonic when used as intended: as a way to avoid name-collisions. If you think it's worthwhile to protect an attribute from being overwritten, you might as well guard against accidental conflict with the underlying name. Here you are assuming that a user of your class could not possibly have a valid reason for getting to the underlying variable. Don't make those decisions for someone else, in Python, we are all adults here. They can use the demangled name of the internal variable if they want access to it. Finally, some people think read-only attributes are unpythonic. I think that's ridiculous, although in general I'd advise against making attributes read-only willy-nilly. But there's a time and place for it. Generally, properties are for doing some form of calculation, not for making things read-only. That might be how properties are generally used, but if for some reason I wanted a read-only attribute, that's how I'd do it. [snip strawman stuff] It is not your job to protect those users who do not use your code properly from themselves; that way lies madness. I'm sorry, but the universe is not as simple as you are making it out to be. Blanket statements like the one you just gave here are not something that should ever be blindly adhered to. If, in my judgment, users would be prone to overwrite one of my attributes, and if I designed the system to rely on that attribute, and if the results of changing it are bad enough, then by golly I'm going to make the attribute harder than usual to modify. And yes, that is my job. Users who want to change it anyway can curse me and then go demangle the name themselves. Carl Banks Thanks for all the replies -- While greatly simplified, the reason for desiring read-only attributes in this case is that the interface is to a remote server whose policy relies on data objects represented by this class to have a few values which never change or are managed by the server and not the end user. Changing the ID value would break things on the server, so I wanted to write the interface class to respect those conventions. I'm well aware that if a developer really wanted to, they could get around it no matter what I did, but I figure that if I at least make it really difficult it will be obvious that they're really going into dangerous territory. Further (and this might just be a tad paranoid), user interface code which might use this API might be dangerous. It's one thing for a developer to break the rules when they need to, but a user shouldn't be able to. By enforcing read-only on the API it ensure (at least in my world view) that a developer writing a user interface against it won't have to code defensively against malicious input. However, since the difference between the two is simply attribute name-mangling it's practically a pedantic issue. I guess there might be some hyper-specific scenario where MyInterface would still be useful, but this one might not be it. Again, thanks. :) -- http://mail.python.org/mailman/listinfo/python-list
unpythonic use of property()?
Consider: code: class MyInterface(object): def __get_id(self): return self.__id id = property(fget=__get_id) def __init__(self, id, foo): self.__id = id self.foo = foo class MyInterface2(object): def __init__(self, id, foo): self._id = id self.foo = foo @property def id(self): return self._id The situation is that an API to an external resource must enforce consistency with the resource's expectations of data (ie: ID's must be immutable). The Python docs clearly show the use of the property() decorator in both the example classes shown here. Both ensure that an assignment to the class objects' 'id' attribute will raise an AttributeError exception. Both will satisfy the requirements. I assume we all know what the implementation differences are between the two examples here. What I am curious about is what use case would the MyInterface example have? My guess is that it's added protection for attribute names on objects generated from factories or generators. I'd like a more concrete explanation from someone who has encountered a scenario where it was required. I was recently informed that it was 'unpythonic' and have since been a little confused by the term. I've heard it bandied about before but never paid much attention. What is 'unpythonic'? What about this example is unpythonic? -- http://mail.python.org/mailman/listinfo/python-list
Re: decorators don't play nice with nose?
hyperboreean hyperbore...@nerdshack.com writes:
From: hyperboreean hyperbore...@nerdshack.com
Subject: decorators don't play nice with nose?
Newsgroups: comp.lang.python
To: python-list@python.org
Date: Mon, 06 Apr 2009 11:01:04 +0300
Hi, I am trying to test the business part of a web service. For this I
am using unittest nose.
I wrote a decorator that should handle the xml test file retrieval,
but it seems I can't get it working with nose.
Here's the code:
* MyApp.py -- base test class *
import os
import unittest
from MyApp.Core import XmlParser
__all__ = ['MyAppTest', 'setup']
PATH = os.path.dirname(__file__) or ''
class setup(object):
Decorator to ease the use of xml files in MyApp tests.
The way it works it that it decorates a test method which has a first
default parameter called 'parser' and it overwrites this parameter value
with a XmlParser instance.
The xml file should be located under:
data/testedBusinessRequest/testMethodName.xml
def __init__(self, testedBusinessRequest = ''):
self.testedBusinessRequest =\
testedBusinessRequest.lower()
def _getXmlParser(self, xml):
documentElement = XmlParser.parseXmlStream(xml)
parser = XmlParser.getParser(documentElement)
return parser
def __call__(self, method):
# TODO: error handling here
methodName = method.func_code.co_name
methodName = methodName.split('_')[1]
xmlFolder = self.testedBusinessRequest
xmlFile = '%s.xml' % methodName
path = os.path.join(PATH, 'data',
xmlFolder, xmlFile)
f = open(path)
xml = f.read()
f.close()
method.func_defaults = (self._getXmlParser(xml),)
return method
class MyAppTest(unittest.TestCase):
def setUp(self):
self.database = Database()
def tearDown(self):
pass
* test_Login.py - test a business request *
from MyAppTest import MyAppTest, setup
from MyApp import Login
class TestLogin(MyAppTest):
testedBusinessRequest = 'Login'
@setup(testedBusinessRequest)
def test_validParameters(self, parser = None):
FWIW, nose and unittest both provide methods for setting up and
tearing down tests. You should probably look at those first before
rolling your own. At the very least it will give you an idea of how
yours should work.
Cheers
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTML Generation
Tino Wildenhain t...@living-examples.com writes:
Hi Mike,
Mike wrote:
Hello all,
I'm writing a web app and wanted to do some html generation (I
really do not like to maintain or write html).
I'm thinking of writing a dsl based on the following:
def html():
return
def a():
return
def body():
return
That would be writing HTML just another way and also
mixing code and representation, this is generally not a
good idea.
If you really don't want to maintain HTML along with
your code, I'd suggest an approach like
Zope Page Templates
http://en.wikipedia.org/wiki/Zope_Page_Templates#Zope_Page_Templates
which can be used outside Zope as well, (even with many other languages)
For example: http://zpt.sourceforge.net/
Regards
Tino
You could also look at CL-WHO for inspiration.
I feel the same way about templates -- the seperation is unnecessary
complexity. One knowledge domain for everything please. We're not
building the tower of Babel here.
I use Python through all of my desktop application development without
switching to markup languages and stylesheets and other scripting
languages (at least VERY rarely).
Why is the web any different?
It's rather somewhat trivial to implement a simple grammar in Python for
describing a page using Python data structures and Pythonic idioms.
I've toyed with it using a base Tag metaclass which creates the
__str__() representation for a Tag-metaclassed object based on the name
of the class.
It would look something like this:
code:
from tags import html, head, meta, title, body, div, p, a
mypage = html(
head(
meta(attrs={'http-equiv': Content-Type,
'content': text/html;}),
title(My Page)),
body(attrs={'background': #ff;},
div(attrs={'id': article-content},
p(a(attrs={'href': http://python.org},
Python.org)
tabbed_file(mypage, open('myfile.html', 'w'))
Unfortunately Python isn't as malleable when it comes to syntax as Lisp,
but I don't think it's a lost endeavour.
Having to know a template language, markup, stylesheets, CSS, plus all
of Python and good web application design is a huge headache and puts
too many cooks in the kitchen. I'd rather 95% of the work be in pure
Python and keep the team small.
Just 0.02 monetary units.
--
http://mail.python.org/mailman/listinfo/python-list
Re: HTML Generation
Stefan Behnel stefan...@behnel.de writes:
J Kenneth King wrote:
from tags import html, head, meta, title, body, div, p, a
mypage = html(
head(
meta(attrs={'http-equiv': Content-Type,
'content': text/html;}),
title(My Page)),
body(attrs={'background': #ff;},
div(attrs={'id': article-content},
p(a(attrs={'href': http://python.org},
Python.org)
tabbed_file(mypage, open('myfile.html', 'w'))
See here for another example that uses lxml.html:
http://codespeak.net/lxml/lxmlhtml.html#creating-html-with-the-e-factory
Stefan
Ah, looks good. Have never used nor finished the example I had given --
only meant as inspiration. I'm not surprised it has been done by someone
else.
I've been thinking about the co-routines presentation recently given at
Pycon and have been thinking about ways they could be used to extend the
grammar a bit.
At present, it looks like in my example and the lxml.html example that
the HTML structures created are declarative. Scripting the generation of
child elements for example would break up the flow...
code:
E.HTML(
E.HEAD(
E.TITLE(Best Page Evar!)
),
E.BODY(
E.H1(E.CLASS(best-articles-heading), Best Articles),
E.UL(E.CLASS(articles))
)
)
for article in articles:
E.HTML.BODY.UL.append(E.LI(article['title]))
... at least that's how I assume it would work.
I think in my prototype you could use list-comprehensions in-line to add
the elements, but it was quite an eyesore.
code:
my_html = html(
head(
title(Best Page Evar!)
),
body(
ul(
[li(article['title']) for article in articles]
)
)
)
I guess it's not that bad, but I had a feeling it could look prettier
like how naturally CL-WHO reads in Lisp. It's just that the list
comprehension doesn't read well in this context; it'd be more natural to
read it as for each article in article, create a list element with the
article title instead.
I get the impression that creating a chain of co-routines would reveal a
grammar for talking about generating web pages. Something like...
code:
html_to_file(
html(
head(
title(Best page evar)
),
body(
h1({'class': underline-heading},
Newest Articles),
unordered_list(
articles,
['title', 'href'],
'%-10s: %s')
)
),
file=sys.stdout
)
Where `unordered_list` would walk the elements in its first argument,
extract the values from the keys specified by its second argument, and
format the results using the string from its third argument and simply
return a ul() object with nested li() objects with all the data inserted
into them.
Of course, this is very off-the-cuff; I only started picking up interest
in this old subject this morning. ;) I could be talking way out of my
ass here. No idea if any of it's even practical.
Anyway -- OP: there are many ways to approach HTML generation and it's a
good pursuit. If you come up with something new and unique, please
share! Down with templates! :)
--
http://mail.python.org/mailman/listinfo/python-list
mocking xmlrpclib
At the risk of sounding like I don't know what I'm doing, I must say that I am finding it rather difficult/tedious to mock the xmlrpclib interface using minimock. I refuse to believe that I'm the only developer to have tried this before, but google isn't being my friend and I can't seem to get it to work outside the interpreter (ie: in actual unit tests). Has anyone in c.l.p encountered this problem or have any tips? -- http://mail.python.org/mailman/listinfo/python-list
Re: Roundup Issue Tracker release 1.4.7
Richard Jones richardjo...@optushome.com.au writes: I'm proud to release version 1.4.7 of Roundup. - Allow CGI frontend to serve XMLRPC requests. - Added XMLRPC actions, as well as bridging CGI actions to XMLRPC actions. Sweet. I'm working on a small project called TracShell which is a command-line front-end to the Trac issue tracker and uses XML-RPC to communicate with the server. However, as I work with it/hack on it I'm slowly building it into a generic front-end to work with any XML-RPC enabled issue tracker. I'll definitely look into extending it to work with Roundup in the future now that it has XML-RPC support. :) Keep up the great work! -- http://mail.python.org/mailman/listinfo/python-list
Re: While loop
Fab86 fabien.h...@gmail.com writes:
On Mar 5, 5:23 pm, Marco Mariani ma...@sferacarta.com wrote:
Fab86 wrote:
Is it possible to get the program to catch the exception, wait 10
seconds, then carry of from where it was rather than starting again?
something like this? probably works in PASCAL as well :)
i=0
while i len(stuff):
try:
do_with(stuff[i])
except SomeError:
sleep(10)
continue
i+=1
using sleep and then continue just makes the search start from the
first search term like before.. Would it be easier to understand if I
posted sections of my code?
i = 0
k = open('blah', 'w')
domains = [au, ca, nl, be, ...]
srch = WebSearch(app_id=YahooKey)
while i200:
try:
for domain in domains:
srch.query = test site:.%s % domain
res = srch.parse_results()
print k, res.total_results_available
i = i + 1
except SearchError:
(I currently close then reopen document here then restart i to 0)
Any ideas?
You should check out the sched module for scheduling tasks.
code
import sched, time
domains = [au, ca, nl, be, ru, us, de]
s = sched.scheduler(time.time, time.time)
def do_query(query, domain):
search.query = %s site:.%s % (query, domain)
try:
result = search.parse_results()
except SearchError, e:
print sys.stderr, e
else:
print k, result
for domain in domains:
s.enter(2, 1, do_query, domain)
s.run()
/code
YMMV
--
http://mail.python.org/mailman/listinfo/python-list
ANN: tracshell 0.1r23
Hello everyone, Just wanted to let everyone know that I'm still working on TracShell and it has come a long way in the last couple of weeks! As of r23, TracShell has introduced the following features and fixes: - TracShell now queries the Trac server for a list of available methods and ensures your shell session only has the corresponding commands. - Command shortcuts (`help shortcuts` to see a list) - SSL support - A slew of bugfixes Just head on over to the project page and grab a fresh copy from the repository: http://code.google.com/p/tracshell Thanks to the contributors who've already helped a lot by submitting patches. Bug reports, feature requests, comments and critiques are encouraged! All the best, J Kenneth King -- http://mail.python.org/mailman/listinfo/python-list
Re: Musings: Using decorators to reduce duplicate exception handling
Cameron Simpson c...@zip.com.au writes: On 17Feb2009 15:12, J Kenneth King ja...@agentultra.com wrote: | I recently started a project called TracShell | (http://code.google.com/p/tracshell) where I make heavy use of the | xmlrpclib core module. | | When the number of RPC calls was small, wrapping each call in try/except | was acceptable. [...] | To combat the duplication, my clever idea was to use a function | decorator to wrap any function that used xmlrpclib calls: | def catch_errors(fn): [...] | Maybe I could rename the decorator to something meaningful, but besides | that it works pretty well. Now any function I write in my class that | uses RPC calls can be wrapped by this decorator and have the exception | handling done in a uniform way for these particular exceptions. My Python fu is still weak, but I had a similar issue last year and wrote a context manager, used thus: with NoExceptions(handler): ... do stuff that may break ... The constructor takes a handleException argument (which may be None, but it has to be explicit) to tune which exceptions are caught; default is everything. This was for a daemon thread that did RPC calls, so it was actually fairly important to never die; normally you'd want to catch only specific exceptions. Code: class NoExceptions(object): ''' A context manager to catch _all_ exceptions and log them. Arguably this should be a bare try...except but that's syntacticly noisy and separates the catch from the top. ''' def __init__(self, handleException): ''' Initialise the NoExceptions context manager. The handleException is a callable which expects (exc_type, exc_value, traceback) and returns True or False for the __exit__ method of the manager. If handleException is None, the __exit__ method always returns True, suppressing any exception. ''' self.__handler = handleException def __enter__(self): pass def __exit__(self, exc_type, exc_value, traceback): if self.__handler is not None: return self.__handler(exc_type, exc_value, traceback) if exc_type is not None: print sys.stderr, ignore %s % (exc_type,) return True It doesn't actually solve your duplication issue and is a bit of a niche application, but it's a lot shorter than an inline try/except and to my eye reads a little better, and it keeps the exception handling up front at the calling end where it is visible. Cheers, Ah, that's pretty interesting. I've found a couple other decorator-style recipes in the cookbooks online (http://code.activestate.com/recipes/408937/). I think you see my point: every single RPC call needs to be caught just in case because they are so volatile -- yet you're always looking for the exact same exception everywhere... too noisy to have inline try/except. I wonder if there are other use cases where Python developers have found the need to abstract away repetitive exception handling. I think this is the first I've seen the newer Python context managers being used (though it sneakily looks a lot like what Lisp calls, unwind-protect). Neat idea. -- http://mail.python.org/mailman/listinfo/python-list
Re: Musings: Using decorators to reduce duplicate exception handling
Gabriel Genellina gagsl-...@yahoo.com.ar writes: En Tue, 17 Feb 2009 21:12:57 -0200, J Kenneth King ja...@agentultra.com escribió: I recently started a project called TracShell (http://code.google.com/p/tracshell) where I make heavy use of the xmlrpclib core module. When the number of RPC calls was small, wrapping each call in try/except was acceptable. However, this obviously will duplicate code all over the place. There are only ever two exceptions that the module itself will throw: xmlrpclib.ProtocolError and xmlrpclib.Fault -- both very useful, but not show stoppers. To combat the duplication, my clever idea was to use a function decorator to wrap any function that used xmlrpclib calls: def catch_errors(fn): A decorator to catch typical xmlrpclib exceptions def wrapped(*args, **kwargs): try: return fn(*args, **kwargs) except xmlrpclib.ProtocolError, e: print There was a problem communicating with the server. print URL: %s % e.url print Headers: %s % e.headers print Error code: %d % e.errcode print Error message: %s % e.errmsg print Please file a report with the TracShell developers. pass except xmlrpclib.Fault, e: print A fault ocurred print Fault code: %d % e.faultCode print Fault string: %s % e.faultString print If you think this message is the result of an error, print please file a report with the TracShell developers. pass return wrapped I don't like the idea of hiding an exception. The caller code doesn't know an exception occurred, and just continue doing its work, with bogus results... is this what you want? Also, you don't include the stack trace - and usually it contains very valuable information. When your users start filing a report with the TracShell developers and you feel clueless, a stack trace is important (you don't have to show it on screen - a log file is even better). If all you want is to customize the error message, use sys.except_hook Looking into the code, those pass statement are useless; and error messages are usually written to stderr instead of stdout. Thanks for the ideas. I haven't actually used this pattern in any projects before, I was just looking for a way to reduce the number of common try/except statements I needed to have in my code; especially when they all looked exactly the same and were catching every single xml-rpc call littered throughout my class. sys.except_hook looks more like what I was aiming for. I like Lisp's restarts, where I can define error condition handlers in one place without having to litter my code with error handling statements. Cheers. -- http://mail.python.org/mailman/listinfo/python-list
Musings: Using decorators to reduce duplicate exception handling
I recently started a project called TracShell (http://code.google.com/p/tracshell) where I make heavy use of the xmlrpclib core module. When the number of RPC calls was small, wrapping each call in try/except was acceptable. However, this obviously will duplicate code all over the place. There are only ever two exceptions that the module itself will throw: xmlrpclib.ProtocolError and xmlrpclib.Fault -- both very useful, but not show stoppers. To combat the duplication, my clever idea was to use a function decorator to wrap any function that used xmlrpclib calls: def catch_errors(fn): A decorator to catch typical xmlrpclib exceptions def wrapped(*args, **kwargs): try: return fn(*args, **kwargs) except xmlrpclib.ProtocolError, e: print There was a problem communicating with the server. print URL: %s % e.url print Headers: %s % e.headers print Error code: %d % e.errcode print Error message: %s % e.errmsg print Please file a report with the TracShell developers. pass except xmlrpclib.Fault, e: print A fault ocurred print Fault code: %d % e.faultCode print Fault string: %s % e.faultString print If you think this message is the result of an error, print please file a report with the TracShell developers. pass return wrapped Maybe I could rename the decorator to something meaningful, but besides that it works pretty well. Now any function I write in my class that uses RPC calls can be wrapped by this decorator and have the exception handling done in a uniform way for these particular exceptions. I was just curious if other Pythonistas out there used the same idea or have a better one. Thoughts? -- http://mail.python.org/mailman/listinfo/python-list
Re: [ANN] TracShell 0.1 released
J Kenneth King ja...@agentultra.com writes: I tend to work a lot with Trac for project management and have always found the browser interface to be a productivity killer. I always wanted a simple command-line interface to Trac, but having never found one I found a little free time and got off my laurels to make one. TracShell 0.1 is an early release, but it works. So far you can only query and view tickets, but planned updates include the obvious ability to create and edit tickets. Future plans will allow browsing of comments, change histories, attachments, and so forth. Please consider it really beta. The code needs a little tidying up around the edges. If you find any bugs, please report them and I'll fix them ASAP. Ideas, suggestions, and contributions are welcome. http://code.google.com/p/tracshell/ Cheers. Just added the ability to create tickets, more features forthcoming. I highly recommend anyone using this tool to stick to the latest svn versions. I'll package the feature-complete stables going forward. Cheers. -- http://mail.python.org/mailman/listinfo/python-list
Re: is there a project running (GUI Builder for Python ) ?
gc_ott...@yahoo.ca writes: ..I come from Delphi, and compared to Delphi, even Visual Studio vanishes ;-) ...I don't even notice the difference between Delphi (which I'm still using) and wxPython. I think this story happened to other people to, so instead of putting a lot of effort in designing and maintaining a GUI builders, it might be better to choose another solution. btw, the idea I used, can be seen here]http://mientki.ruhosting.nl/data_www/pylab_works/pw_gui_support.html and the code can be found herehttp://code.google.com/p/pylab-works/downloads/list cheers, Stef You know, 10 or more years ago both Borland and Microsoft got it right when they incorporated a GUI with an IDE in their Delphi and Visual Basic products. As wonderful as the Python language is, it is very much a work in progress when compared to the ease of use of the aforementioned products. These products revolutionized the industry with their Rapid Applications Development (RAD). Python reminds me of a toolbox filled with a multitude of single use tools when all you need is a Skilsaw and a combination screwdriver. Gord ... So use the combination screwdriver. Python isn't all things to all people. It is what it is. -- http://mail.python.org/mailman/listinfo/python-list
[ANN] TracShell 0.1 released
I tend to work a lot with Trac for project management and have always found the browser interface to be a productivity killer. I always wanted a simple command-line interface to Trac, but having never found one I found a little free time and got off my laurels to make one. TracShell 0.1 is an early release, but it works. So far you can only query and view tickets, but planned updates include the obvious ability to create and edit tickets. Future plans will allow browsing of comments, change histories, attachments, and so forth. Please consider it really beta. The code needs a little tidying up around the edges. If you find any bugs, please report them and I'll fix them ASAP. Ideas, suggestions, and contributions are welcome. http://code.google.com/p/tracshell/ Cheers. -- http://mail.python.org/mailman/listinfo/python-list
Re: TracShell 0.1 released
Krzysztof Retel krzysztof.re...@googlemail.com writes: On 12 Feb, 14:06, J Kenneth King ja...@agentultra.com wrote: I tend to work a lot with Trac for project management and have always found the browser interface to be a productivity killer. I always wanted a simple command-line interface to Trac, but having never found one I found a little free time and got off my laurels to make one. TracShell 0.1 is an early release, but it works. So far you can only query and view tickets, but planned updates include the obvious ability to create and edit tickets. Future plans will allow browsing of comments, change histories, attachments, and so forth. Please consider it really beta. The code needs a little tidying up around the edges. If you find any bugs, please report them and I'll fix them ASAP. Ideas, suggestions, and contributions are welcome. http://code.google.com/p/tracshell/ Cheers. Nice work. Maybe it would be worth building at some point vim or emacs interface. Krzysztof Thanks. :) I considered a native editor interface, but since both vi and emacs can run a shell within a buffer, I decided to go with a more flexible solution. However, if someone wanted to, the actual code fetching and displaying the data isn't tied directly to the shell interface. It could be possible to dispatch calls from a native editor front-end if it doesn't mind spawning a python interpreter (or keeping one running) for each request. Anyway, thanks for checking it out. Hope you find it useful. I do. :) -- http://mail.python.org/mailman/listinfo/python-list
Re: is there a project running (GUI Builder for Python ) ?
azrael jura.gro...@gmail.com writes: To be honest, in compare to Visual Studio, Gui Builders for wx widgets are really bad. That's because Visual Studio is a Microsoft product to build interfaces for Microsoft products. wx on the other hand is cross platform and ergo, much more complicated. Do you know if there is any project curently running for GUI builders, maybe for WX, That I could maybe join. You could help wx. Make your own gui builder if you think you could do better. There's also GTK and Qt... and probably many others. -- http://mail.python.org/mailman/listinfo/python-list
Re: [Web 2.0] Added-value of frameworks?
Bruno Desthuilliers bdesth.quelquech...@free.quelquepart.fr writes: Gilles Ganault a écrit : Hello If I wanted to build some social web site such as Facebook, what do frameworks like Django or TurboGears provide over writing a site from scratch using Python? Quite a lot of abstractions and factorisation of the boilerplate code, a known way to organize your application, and possibly a good integration of the usual components (and wrt/ Django, a customizable yet fairly usable OOTB admin interface). For simple to mediumly complex applications, this can mean more than 80% of the grunt work. The counterpart is mostly learning and understanding the framework, which means you may not save that much time on a first project - but it can really pay off then. One of my coworker started this morning a (really simple...) project which acceptance tests are to begin on monday, and we are all quite confident he'll deliver on time, thanks to Django. Well the big negative is when you application design starts expanding past the framework design. A percolator makes a decent cup of coffee but it really isn't meant for making fancy lattes. This is where the 90/10 rule will catch you if you're not careful. 90% of the effort will be spent on the last 10% of the work. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flattening lists
mk mrk...@gmail.com writes: Hello everybody, Any better solution than this? def flatten(x): res = [] for el in x: if isinstance(el,list): res.extend(flatten(el)) else: res.append(el) return res a = [1, 2, 3, [4, 5, 6], [[7, 8], [9, 10]]] print flatten(a) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] Regards, mk http://mail.python.org/pipermail/python-list/2005-July/330367.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Added-value of frameworks?
Matimus mccre...@gmail.com writes: On Feb 4, 8:08 am, Gilles Ganault nos...@nospam.com wrote: Hello If I wanted to build some social web site such as Facebook, what do frameworks like Django or TurboGears provide over writing a site from scratch using Python? Thank you for your feedback. Why not just look at the frameworks themselves and see what they have to offer. Django and Turbogears both have pretty good tutorials. You can be up and running in 20 minutes with each one. You be the judge. The frameworks provide a lot of boilerplate code that I would rather not write. They are probably more secure and scalable than something I would come up with. You also get many extras for free. I think in both of the frameworks you mention you get an administrative back end for free. Other people have created apps/plugins that you can use with those frameworks. So, for example, you might be able to borrow the code to help you add a forum to your site. I'm not sure I know the advantage of not using a framework. Unless I get to write more code is an advantage. Creating your own framework might be fun, but if you really just want a website don't do more work than you need to. Matt In other words, it boils down to what you get paid to do. If you're being paid for a frob (the product, in this case a website) then you use a frob-maker (a framework). If you're being paid to make frobs, then you make the frob-maker. Most frob-makers are good at producing frobs of a certain kind. Just choose the frob-maker that makes the frobs you need. In rare circumstances you'll need a really customized frob. Call on me when you get there. ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: What is wrong in my list comprehension?
Chris Rebert c...@rebertia.com writes: Python 2.6 (r26:66714, Nov 18 2008, 21:48:52) [GCC 4.0.1 (Apple Inc. build 5484)] on darwin Type help, copyright, credits or license for more information. bool(-1) True str.find() returns -1 on failure (i.e. if the substring is not in the given string). -1 is considered boolean true by Python. That's an odd little quirk... never noticed that before. I just use regular expressions myself. Wouldn't this be something worth cleaning up? It's a little confusing for failure to evaluate to boolean true even if the relationship isn't direct. -- http://mail.python.org/mailman/listinfo/python-list
Re: What is wrong in my list comprehension?
Chris Rebert c...@rebertia.com writes: Python 2.6 (r26:66714, Nov 18 2008, 21:48:52) [GCC 4.0.1 (Apple Inc. build 5484)] on darwin Type help, copyright, credits or license for more information. bool(-1) True str.find() returns -1 on failure (i.e. if the substring is not in the given string). -1 is considered boolean true by Python. That's an odd little quirk... never noticed that before. I just use regular expressions myself. Wouldn't this be something worth cleaning up? It's a little confusing for failure to evaluate to boolean true even if the relationship isn't direct. -- http://mail.python.org/mailman/listinfo/python-list
Re: What is wrong in my list comprehension?
Stephen Hansen apt.shan...@gmail.com writes: str.find() returns -1 on failure (i.e. if the substring is not in the given string). -1 is considered boolean true by Python. That's an odd little quirk... never noticed that before. I just use regular expressions myself. Wouldn't this be something worth cleaning up? It's a little confusing for failure to evaluate to boolean true even if the relationship isn't direct. But what would you clean it up to? str.find can return 0 ... which is a *true* result as that means it finds what you're looking for at position 0... but which evaluates to boolean False. The fact that it can also return -1 which is the *false* result which evaluates to boolean True is just another side of that coin. What's the options to clean it up? It can return None when it doesn't match and you can then test str.find(a) is None... but while that kinda works it also goes a bit against the point of having boolean truth/falsehood not representing success/failure of the function. 0 (boolean false) still is a success. Raising an exception would be a bad idea in many cases, too. You can use str.index if that's what you want. So there's not really a great solution to cleaning it up . I remember there was some talk in py-dev of removing str.find entirely because there was no really c, but I have absolutely no idea if they ended up doing it or not. --S (Sorry all for the multiple post... my gnus fudged a bit there) That's the funny thing about integers having boolean contexts I guess. Here's a case where 0 actually isn't False. Any returned value should be considered True and None should evaluate to False. Then the method can be used in both contexts of logic and procedure. (I guess that's how I'd solve it, but I can see that implementing it is highly improbable) I'm only curious if it's worth cleaning up because the OP's case is one where there is more than one way to do it. However, that's not the way the world is and I suppose smarter people have discussed this before. If there's a link to the discussion, I'd like to read it. It's pedantic but fascinating no less. -- http://mail.python.org/mailman/listinfo/python-list
Re: What is wrong in my list comprehension?
Chris Rebert c...@rebertia.com writes: Python 2.6 (r26:66714, Nov 18 2008, 21:48:52) [GCC 4.0.1 (Apple Inc. build 5484)] on darwin Type help, copyright, credits or license for more information. bool(-1) True str.find() returns -1 on failure (i.e. if the substring is not in the given string). -1 is considered boolean true by Python. That's an odd little quirk... never noticed that before. I just use regular expressions myself. Wouldn't this be something worth cleaning up? It's a little confusing for failure to evaluate to boolean true even if the relationship isn't direct. -- http://mail.python.org/mailman/listinfo/python-list
Re: What is wrong in my list comprehension?
Chris Rebert c...@rebertia.com writes: Python 2.6 (r26:66714, Nov 18 2008, 21:48:52) [GCC 4.0.1 (Apple Inc. build 5484)] on darwin Type help, copyright, credits or license for more information. bool(-1) True str.find() returns -1 on failure (i.e. if the substring is not in the given string). -1 is considered boolean true by Python. That's an odd little quirk... never noticed that before. I just use regular expressions myself. Wouldn't this be something worth cleaning up? It's a little confusing for failure to evaluate to boolean true even if the relationship isn't direct. -- http://mail.python.org/mailman/listinfo/python-list
Re: Recommendation for a small web framework like Perl's CGI::Application to run as CGI?
excord80 excor...@gmail.com writes: I need to make a small, relatively low-traffic site that users can create accounts on and log into. Scripts must run as cgi (no mod_python or FastCGI is available). Can anyone recommend a small and simple web framework for Python, maybe similar to Perl's CGI::Application? Or would it just be better to roll my own? web.py pretty simple. lots of deployment options. ends up building into your own framework as your application grows. -- http://mail.python.org/mailman/listinfo/python-list
Re: I'm a python addict !
Linuxguy123 linuxguy...@gmail.com writes: I just started using python last week and I'm addicted. I hate Perl. I never did learn to use it with any competence. I has to be the most obfuscated, cryptic language I've ever seen. Making it object oriented only makes it worse ! .. snip .. I program full-time in Python, so I share your excitement and enthusiasm. But bashing Perl like that doesn't make you sound very smart. I'm probably one of the very few Python programmers who came from (and still occassionally) use Perl. I've written non-trivial programs in it and from my experience I can tell you that it's actually a great language. The Moose object system is well beyond Python's class system. But I guess you wouldn't know that. So yay for Python, but don't get in the habit of criticising that which you do not know. -- http://mail.python.org/mailman/listinfo/python-list
Re: I'm a python addict !
J Kenneth King ja...@agentultra.com writes: Linuxguy123 linuxguy...@gmail.com writes: I just started using python last week and I'm addicted. I hate Perl. I never did learn to use it with any competence. I has to be the most obfuscated, cryptic language I've ever seen. Making it object oriented only makes it worse ! .. snip .. I program full-time in Python, so I share your excitement and enthusiasm. But bashing Perl like that doesn't make you sound very smart. I'm probably one of the very few Python programmers who came from (and still occassionally) use Perl. I've written non-trivial programs in it and from my experience I can tell you that it's actually a great language. The Moose object system is well beyond Python's class system. But I guess you wouldn't know that. So yay for Python, but don't get in the habit of criticising that which you do not know. I just realized this might become flame-bait. Please disregard the flamey bits. :( -- http://mail.python.org/mailman/listinfo/python-list
Re: Two questions about style and some simple math
Spoofy spoo...@gmx.net writes:
.. snip ..
2.
For maintaining the character attributes I creates a seperate class. I
wonder weather this is an overuse of OO (instead of just making the
attributes plain variables of the Char class) and if the way I wrote
this is OK (somehow this looks cool to me but maybe too showy?)
class Attributes(object):
ATTRIBUTES = {attack: 0, defence: 0, ability: 0, courage:
0, condition: 0}
def __init__(self, **kwargs):
self.__dict__.update(self.ATTRIBUTES)
for arg in kwargs:
if arg not in self.ATTRIBUTES:
raise ValueError(Unkown character attribute '%s' %
arg)
self.__dict__[arg] = kwargs[arg]
Again, I appreciate any tips. I you need more code (for the bigger
picture or such), just ask.
Thanks in advance
I think the first part has been covered well.
If you want an opinion, this class isn't showy at all, rather it is
ugly and unnecessary.
Firstly it's bad because:
1. ATTRIBUTES is still modifiable after insantiation. This breaks the
restriction you're expressing in __init__
2. You want to avoid modifying __dict__ directly if you can. It bypasses
the whole point of using named attributes.
What you'd really want in a case where a class has a restricted set of
attributes is __slots__. Classes defined with it have no __dict__ and
thus attributes cannot be dynamically added to them after
instanciation (a __slots__ defined class will raise an exception in this
case).
However, even in this case, it doesn't make sense to encapsulate the
attributes of your game's Character objects in this way. Your Hero
class should have its attributes directly associated to it: Hero.health,
Hero.defence, and so forth. If you want to encapsulate a common set of
attributes available to a class of objects, you'd create a Character
class with those general attributes, sub-class your NPC's and Hero from
it, and specialize those sub-classes as needed.
HTH,
j_king
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pyro deadlock
MatthewS schaefer...@gmail.com writes: I'd like to know if the following behavior is expected and can be avoided: I have a Pyro server object that maintains a queue of work, and multiple Pyro worker objects that take work off the queue by calling a method on the server (get_work) and then return the work to the server by calling another method (return_result). The problem is that I experience a deadlock condition in the return_result method. Is this deadlock expected or should Pyro be able to synchronize the remote calls to server's callback method when called from multiple workers at the same time? If so any hints are appreciated. /Matthew Depends on how you're implementing the work/result queues. The Queue module is a good start. -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for tips on how to implement ruby-style Domain Specific Language in Python
Jonathan Gardner jgard...@jonathangardner.net writes: On Jan 7, 9:16 am, Chris Mellon arka...@gmail.com wrote: The OP wants a Ruby-style DSL by which he means something that lets me write words instead of expressions. The ruby syntax is amenable to this, python (and lisp, for that matter) syntax is not and you can't implement that style of internal DSL in those languages. The answer to the OP is you can't - use Ruby or modify your requirements. As far as putting the code into Python, yeah, you can't put it in Python. The best you can do is store it in a string and then interpret the string with some function later on. That's what I'm saying. It seems we're defining DSL in two different ways. You can't write a DSL in Python because you can't change the syntax and you don't have macros. You can write a compiler in Python that will compile your DSL. As another poster mentioned, eventually PyPy will be done and then you'll get more of an in-Python DSL. -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for tips on how to implement ruby-style Domain Specific Language in Python
Kay Schluehr kay.schlu...@gmx.net writes: On 8 Jan., 16:25, J Kenneth King ja...@agentultra.com wrote: As another poster mentioned, eventually PyPy will be done and then you'll get more of an in-Python DSL. May I ask why you consider it as important that the interpreter is written in Python? I don't think it's important for Python to have a meta-circular interpreter (though it can't hurt). I see no connection between PyPy and syntactical Python extensions and the latter isn't an objective of PyPy. You can write Python extensions with virtually any Python aware parser. M.A.Lemburg already mentioned PLY and PLY is used for Cython. Then there is ANTLR which provides a Python grammar. I also know about two other Python aware parsers. One of them was written by myself. Because... there is no connection to see? I never mentioned any such relation. DSL's tend to be a natural side-effect of languages which can manipulate their own expressions without extensive parsing. Creating a new parser that can generate Python AST's is certainly a valid approach (and probably the easiest one). It's not the only one. It depends on your definition of a DSL. My definition isn't satisfied with creating a parser, and so my answers reflect that. -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for tips on how to implement ruby-style Domain Specific Language in Python
Jonathan Gardner jgard...@jonathangardner.net writes: On Jan 6, 12:24 pm, J Kenneth King ja...@agentultra.com wrote: Jonathan Gardner jgard...@jonathangardner.net writes: On Jan 6, 8:18 am, sturlamolden sturlamol...@yahoo.no wrote: On Jan 6, 4:32 pm, mark mark.fi...@googlemail.com wrote: I want to implement a internal DSL in Python. I would like the syntax as human readable as possible. Also beware that Python is not Lisp. You cannot define new syntax (yes I've seen the goto joke). This isn't really true. You can, for instance, write a program (in Python) that takes your pseudo-Python and converts it into Python. This is what a number of templating libraries such as Mako do. Which is not even close to being the same. Lisp - the program source is also the data format Python - the program source is a string I could go on a really long rant about how the two are worlds apart, but I'll let Google tell you if you're really interested. I get that Lisp is special because you can hack on the reader as it is reading the file in. This is strongly discouraged behavior, as far as I know, despite the number of cute hacks you can accomplish with it. It is generally discouraged unless there's a reason for it. But consider that this really isn't different than having a program read in the lisp-with-modification source and spitting out pure lisp, to be read by an honest-to-gosh lisp program later. If that's the case, then Lisp and Python really aren't that different in this regard, except that you don't have the option of modifying the reader as it reads in the file. I think you are missing the distinction. Lisp expressions are also data structures. A Lisp expression can be passed to functions and macros to be operated on before being executed. When you're writing Lisp source, you're basically looking at the AST on one level and when you start writing macros for your program, you're creating a DSL or interface to that AST. Lisp source is eventually expanded to a giant list that is consed by the evaluator (as far as I understand it. I'm just getting into the compiler stuff myself). Consider: (my-func 1 2 3) This is just a list, the primitive data-type in Lisp! This piece of data can be operated on by other bits of Lisp code because it is just a list as far as Lisp is concerned. In contrast, Python source is a string that needs to be parsed into bytecode which is then run through the interpreter. The AST is completely hidden from the source author. Python expressions are not data types either and hence no macros -- I can't write a python function that generates python code at compile time. I can only write a python program that parses some other string and generates code that can be run by another interpreter. Consider: for i in range(0, 100): do_something_interesting(i) That's a pretty straight forward Python expression, but I can't do anything with it -- it's not a unit of data, it's a string. The distinction is not subtle by any means. -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for tips on how to implement ruby-style Domain Specific Language in Python
Kay Schluehr kay.schlu...@gmx.net writes: On 7 Jan., 16:50, J Kenneth King ja...@agentultra.com wrote: Python expressions are not data types either and hence no macros -- I can't write a python function that generates python code at compile time. Have you ever considered there are languages providing macros other than Lisp? Of course. Macros have nothing to do with homoiconcity. Not directly, no. I can only write a python program that parses some other string and generates code that can be run by another interpreter. No, it is the same interpreter and it is also possible to modify python parsers on the fly. This is just not possible with Pythons builtin parser. PyPy is probably the best bet when/if it gets finished. Consider: for i in range(0, 100): do_something_interesting(i) That's a pretty straight forward Python expression, but I can't do anything with it -- it's not a unit of data, it's a string. The distinction is not subtle by any means. -- http://mail.python.org/mailman/listinfo/python-list
Re: Noob question: Is all this typecasting normal?
Gabriel Genellina gagsl-...@yahoo.com.ar writes:
En Mon, 05 Jan 2009 02:03:26 -0200, Roy Smith r...@panix.com escribió:
The other day, I came upon this gem. It's a bit of perl embedded in a
Makefile; this makes it even more gnarly because all the $'s get
doubled to
hide them from make:
define absmondir
$(shell perl -e ' \
sub absmon { my $$a = $$_[0]; \
if ( $$^O =~ m/cygwin|MSWin32/i ) {
$$prefix = `/bin/mount -p|awk NR==2{print
\\\$$1}`;
chomp($$prefix); \
$$a = ($$_[1]||$(PWD)) . /$$a \
unless ( $$a =~ m !^(:?$$prefix|/|[A-Za-z]:)! ); \
} else { $$a = ($$_[1]||$(PWD)) . /$$a unless ( $$a =~ m !^/!
); } \
return unslash(undot(undotdot($$a))); }; \
sub unslash ($$) { $$_[0] =~ s://+:/:g; $$_[0] =~ s:/$$::;
return($$_[0]);
}; \
sub undot ($$) { $$_[0]=~s:/\./:/:g; return ($$_[0]); }; \
sub undotdot ($$) { my $$in = $$_[0]; \
return ( $$in =~ s:/[^/.][^/]*/\.\.::g )?undotdot($$in):$$in; }; \
print absmon($(1),$(2)); \
' )
endef
Barf-o-rama. I know what it's supposed to do, and I still can't
figure it out.
Ouch! Me too, when I come to some piece of Perl code I've written some
years ago, I invariably think what's all this noise?. Never happens
with other languages I've used in the past.
I still occassion upon the chance to write some Perl, and even as a
full-time developer who works with Python for a living, I relish every
opportunity.
The funny thing is that I've never had the problem of writing code like
this in Perl. The example is a very poor use-case and doesn't reflect on
the useful/useless-ness of the language itself but more on the choices
of the implementor.
Perl is a very useful language overall and when used properly, very
powerful.
--
http://mail.python.org/mailman/listinfo/python-list
Re: image recogniton?
Li Han lihang9...@gmail.com writes: Hi! I know little about the computer image processing, and now I have a fancy problem which is how to read the time from the picture of a clock by programming ? Is there anyone who can give me some suggestions? Thank! Li Han I do work in object recognition, and I would classify this as a rather difficult problem. Not impossible of course, but you'll need some OCR to read the clock face and some sort of magnitude vector feature to tell which hand is which and the general direction is is pointing in. Also, depends if we're talking digital clocks or analog. :) The other problem is one of accuracy: depending on your input image, even slight variances can change your results. I'm curious as to what application the solution to this problem is practical for all of its difficulty? -- http://mail.python.org/mailman/listinfo/python-list
Re: looking for tips on how to implement ruby-style Domain Specific Language in Python
Jonathan Gardner jgard...@jonathangardner.net writes: On Jan 6, 8:18 am, sturlamolden sturlamol...@yahoo.no wrote: On Jan 6, 4:32 pm, mark mark.fi...@googlemail.com wrote: I want to implement a internal DSL in Python. I would like the syntax as human readable as possible. Also beware that Python is not Lisp. You cannot define new syntax (yes I've seen the goto joke). This isn't really true. You can, for instance, write a program (in Python) that takes your pseudo-Python and converts it into Python. This is what a number of templating libraries such as Mako do. Which is not even close to being the same. Lisp - the program source is also the data format Python - the program source is a string I could go on a really long rant about how the two are worlds apart, but I'll let Google tell you if you're really interested. -- http://mail.python.org/mailman/listinfo/python-list
Re: If your were going to program a game...
Tokyo Dan huff...@tokyo.email.ne.jp writes: If your were going to program a game in python what technologies would you use? The game is a board game with some piece animations, but no movement animation...think of a chess king exploding. The game runs in a browser in a window of a social site built around the game. The social site has login, chat, player stats, list of active games, etc. AND there is also be a desktop client that accesses the game server via the same communication mechanism (like an AIR-based desktop client/ app) as the browser-based version - I guess using JSON/RPC. Ever see chess.com? I don't know what they're using in the backend, but the client is entirely javascript. You could probably roll your own python javascript compiler to suit your needs. It could probably even build up your own DSL for writing these games. It's a worthwhile project and I think there might be support for it from other developers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better algorithm?
Kottiyath n.kottiy...@gmail.com writes: I have the following list of tuples: L = [(1, 2), (3, 4, 5), (6, 7)] I want to loop through the list and extract the values. The only algorithm I could think of is: for i in l: ... u = None ... try: ... (k, v) = i ... except ValueError: ... (k, u, v) = i ... print k, u, v - 1 None 2 3 4 5 6 None 7 - But, this algorithm doesnt look very beautiful - like say - for k, u, v in L: Can anyone suggest a better algorithm to get the values? Just a note: this isn't really an algorithm problem. ;) It's more of a grammar obstruction. To make your code more simple, it would be nice if the assignment operator would return, None, in the case where there are too few values to unpack from the right-operand of the assignment operator. Aside from the typical workarounds that first came to mind, I started wondering whether it was possible to expose the problem and solve it directly. Sadly, it appears to be non-trivial (or at least, well hidden from the unwashed masses). I'd be really curious if the unpacking machinery were exposed to the developer. I started poking around the operator and types modules, but the implementation isn't obvious. What methods are being called on the operands by the assignment operator in the following statement: a, b, c = some_tuple I'm sorry if this line of inquiry is not very pythonic; but one is curious if this is some python magic happening here. After using the idiom for years I hadn't really thought about it much until recently. -- http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better algorithm?
Paul Rubin http://phr...@nospam.invalid writes: Kottiyath n.kottiy...@gmail.com writes: I have the following list of tuples: L = [(1, 2), (3, 4, 5), (6, 7)] I want to loop through the list and extract the values. Others have suggested messy ways to code what you're asking. At another level, that list format seems like a code smell. You may be better off organizing the program so that L = [(1, None, 2), (3, 4, 5), (6, None, 7)] after which unpacking becomes trivial. Very true. +1 -- http://mail.python.org/mailman/listinfo/python-list
Re: Are Django/Turbogears too specific?
Gilles Ganault nos...@nospam.com writes: Hi I'd like to rewrite a Web 2.0 PHP application in Python with AJAX, and it seems like Django and Turbogears are the frameworks that have the most momentum. I'd like to use this opportunity to lower the load on servers, as the PHP application wasn't built to fit the number of users hammering the servers now. I'm concerned, though, that these frameworks they may be too specific to the tasks they were originally developped for (news articles, AFAIK). Do you think I should just use eg. CherryPy and some basic AJAX? Thank you for any feedback. They're not specific in the sense that they only build certain types of applications. However, they do package their own batteries and expect applications to be designed a certain way. As long as you drink the kool-aid, everything is smooth sailing. That's what any framework banks on -- being useful to 85% of the web applications; not all. Even if they advertise themselves as such, it's just not true. My suggestion is web.py -- It too has its own set of conventions and philosophies, but in my experience it is the most loosely coupled in terms of internal dependencies. Import exactly what you want to use and deploy it the way that suits you. It gives you batteries but you have to pick and choose how to put the pieces together. Therefore, it takes a little longer to get running, but IMO that is the most flexibility you can ask for without writing your own servers and frameworks. Cheers. -- http://mail.python.org/mailman/listinfo/python-list
Re: pydb 1.24
[EMAIL PROTECTED] (R. Bernstein) writes: This release is to clear out some old issues. It contains some bugfixes, document corrections, and enhancements. Tests were revised for Python 2.6 and Python without readline installed. A bug involving invoking from ipython was fixed. The frame command is a little more like gdb's. Exceptions are now caught in runcall(). This is the last release contemplated before a major rewrite. download: https://sourceforge.net/project/showfiles.php?group_id=61395package_id=175827 bug reports: https://sourceforge.net/tracker/?group_id=61395atid=497159 I watched the demo video, look forward to working with it. Any links to that emacs front-end being used in the video? Cheers and thanks! -- http://mail.python.org/mailman/listinfo/python-list
