Re: Using the Python Interpreter as a Reference
On Nov 27, 6:55 pm, Steven D'Aprano steve
+comp.lang.pyt...@pearwood.info wrote:
On Sun, 27 Nov 2011 14:21:01 -0800, Travis Parks wrote:
Personally, I find a lot of good things in Python. I thinking tabs are
out-of-date. Even the MAKE community wishes that the need for tabs would
go away and many implementations have done just that.
Tabs have every theoretical advantage and only one practical
disadvantage: the common toolsets used by Unix programmers are crap in
their handling of tabs, and instead of fixing the toolsets, they blame
the tabs.
The use of spaces as indentation is a clear case of a technically worse
solution winning over a better solution.
I have been
seriously debating about whether to force a specific number of spaces,
such as the classic 4, but I am not sure yet. Some times, 2 or even 8
spaces is appropriate (although I'm not sure when).
Why on earth should your language dictate the width of an indentation? I
can understand that you might care that indents are consistent within a
single source code unit (a file?), but anything more than that is just
obnoxious.
I have always found the standard library for Python to be disjoint. That
can be really beneficial where it keeps the learning curve down and the
size of the standard modules down. At the same time, it means
re-learning whenever you use a new module.
I know what disjoint means, but I don't understand what you think it
means for a software library to be disjoint. I don't understand the rest
of the paragraph.
My language combines generators and collection initializers, instead of
creating a whole new syntax for comprehensions.
[| for i in 0..10: for j in 0.10: yield return i * j |]
Are we supposed to intuit what that means?
Is | a token, or are the delimiters [| and |] ?
Is there a difference between iterating over 0..10 and iterating over
what looks like a float 0.10?
What is yield return?
Lambdas and functions are the same thing in my language, so no need for
a special keyword.
That does not follow. Lambdas and def functions are the same thing in
Python, but Python requires a special keyword.
I also distinguish between initialization and
assignment via the let keyword.
What does this mean? I can guess, but I might guess wrong.
Also, non-locals do not need to be
defined explicitly, since the scoping rules in Unit are far more anal.
What does this mean? I can't even guess what you consider more anal
scoping rules.
In reality though, it takes a certain level of arrogance to assume that
any language will turn out without bumps. It is like I was told in
college long ago, Only the smallest programs are bug free. I think the
same thing could be said for a language. The only language without flaws
would be so small that it would be useless.
I'm pretty sure that being so small that it is useless would count as a
flaw.
What does it mean to say that a language is small?
A Turing Machine is a pretty small language, with only a few
instructions: step forward, step backwards, erase a cell, write a cell,
branch on the state of the cell. And yet anything that can be computed,
anything at all, can be computed by a Turning Machine: a Turing Machine
can do anything you can do in C, Lisp, Fortran, Python, Java... and very
probably anything you can (mentally) do, full stop. So what does that
mean about small languages?
On the other hand, take Epigram, a functional programming language:
http://en.wikipedia.org/wiki/Epigram_(programming_language)
It is *less* powerful than a Turing Machine, despite being far more
complex. Similarly languages like regular expressions, finite automata
and context-free grammers are more complex, bigger, possibly with
dozens or hundreds of instructions, and yet less powerful. Likewise for
spreadsheets without cycles.
Forth is much smaller than Java, but I would say that Forth is much, much
more powerful in some sense than Java. You could write a Java compiler in
Forth more easily than you could write a Forth compiler in Java.
--
Steven
Yes. I was mostly rambling. More explanation would have meant more
typing.
Languages that use type inference heavily typically find unique ways
of indicating literals, including numbers and collections. In Unit,
[||] indicates fixed length arrays, [] is for dynamic arrays, {} is
for sets and unordered dictionaries (at least one value is needed). A
frozen set might be represented with a {||} in the future. I stole the
syntax from F#. Numbers have a trailing letter indicating their types:
signed byte (y), short (s), long (l), float (f), fixed (m), arbitrary
precision (i), rational (r) and imaginary (j). Unsigned versions have
a u before the letter (uy).
Also, .. is the range operator in Unit. 0..10 means 0 through 10 and
0.1 means a floating point number (I missed a dot). 0..10..2 means 0,
2, 4, 6, 8, 10. Saves me from needing a range function, like Python.
Quite a few
Re: Using the Python Interpreter as a Reference
On Nov 28, 2:32 pm, Ian Kelly ian.g.ke...@gmail.com wrote: On Sun, Nov 27, 2011 at 4:55 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: My language combines generators and collection initializers, instead of creating a whole new syntax for comprehensions. [| for i in 0..10: for j in 0.10: yield return i * j |] Are we supposed to intuit what that means? Is | a token, or are the delimiters [| and |] ? Is there a difference between iterating over 0..10 and iterating over what looks like a float 0.10? What is yield return? I would assume that yield return is borrowed from C#, where it is basically equivalent to Python's yield statement. The advantage of using two keywords like that is that you can compare the statements yield return foo and yield break, which is a bit clearer than comparing the equivalent yield foo and return. Having to type out yield return in every comprehension seems a bit painful to me, but I can understand the approach: what is shown above is a full generator, not a single generator expression like we use in Python, so the statement keywords can't be omitted. It's trading off convenience for expressiveness (a bad trade-off IMO -- complex generators should be named, not anonymous). Lambdas and functions are the same thing in my language, so no need for a special keyword. That does not follow. Lambdas and def functions are the same thing in Python, but Python requires a special keyword. I think the implication is that Unit has only one syntax for creating functions, which is lambda-style. In any case, why does Python require a special keyword? def is only used in a statement context, and lambda is only used in an expression context. Why not use the same keyword for both? I think the answer is historical: def came first, and when anonymous functions were added it didn't make sense to use the keyword def for them, because def implies a name being defined. Cheers, Ian Most languages I have worked with have a lambda syntax and a function syntax. It has always been a historical artifact. Languages start out avoiding functional features and then eventually adopt them. It seems that eventually, convenient high-order functions become a must-have (most standard algorithm packages). It is a conflict between old C-style programming and the need for functional code. As soon as functions can be assigned to variables, the code starts looking oddly like JavaScript. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using the Python Interpreter as a Reference
On Nov 28, 3:40 pm, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: Travis Parks wrote: I thinking tabs are out-of-date. Even the MAKE community wishes that the need for tabs would go away The situation with make is a bit different, because it *requires* tabs in certain places -- spaces won't do. Python lets you choose which to use as long as you don't mix them up, and I like it that way. let Parse = public static method (value: String) throws(FormatException UnderflowException OverflowException) Checked exceptions? I fear you're repeating a huge mistake going down that route... -- Greg Exception handling is one of those subjects few understand and fewer can implement properly in modern code. Languages that don't support exceptions as part of their signature lead to capturing generic Exception all throughout code. It is one of those features I wish .NET had. At the same time, with my limited experience with Java, it has been a massive annoyance. Perhaps something to provide or just shut off via a command line parameter. What indications have there been that this has been a flaw? I can see it alienating a large group of up- and-coming developers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using the Python Interpreter as a Reference
On Nov 28, 5:24 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: On Mon, 28 Nov 2011 12:32:59 -0700, Ian Kelly wrote: On Sun, Nov 27, 2011 at 4:55 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: [...] Lambdas and functions are the same thing in my language, so no need for a special keyword. That does not follow. Lambdas and def functions are the same thing in Python, but Python requires a special keyword. I think the implication is that Unit has only one syntax for creating functions, which is lambda-style. In any case, why does Python require a special keyword? def is only used in a statement context, and lambda is only used in an expression context. Why not use the same keyword for both? Because the syntax is completely different. One is a statement, and stands alone, the other is an expression. Even putting aside the fact that lambda's body is an expression, and a def's body is a block, def also requires a name. Using the same keyword for both would require special case reasoning: sometimes def is followed by a name, sometimes without a name. That's ugly. def name(args): block # okay funcs = [def args: expr, # okay so far def name(args): expr, # not okay def: expr, # okay ] def: expr # also okay def: expr expr # but not okay x = def x: expr # okay x = def x(x): expr # not okay Using the same keyword for named and unnamed functions is, in my opinion, one of those foolish consistencies we are warned about. When deciding on syntax, the difference between anonymous and named functions are more significant than their similarities. A good example I have run into is recursion. When a local function calls itself, the name of the function may not be part of scope (non- local). Languages that support tail-end recursion optimization can't optimize. In order to support this, a function in Unit will have access to its own name and type. In other words, special scoping rules are in place in Unit to allow treating a function as an expression. I think the answer is historical: def came first, and when anonymous functions were added it didn't make sense to use the keyword def for them, because def implies a name being defined. That reasoning still applies even if they were added simultaneously. Lambda is pretty old: it certainly exists in Python 1.5 and almost certainly in 1.4. While it doesn't exist as a keyword in Python 0.9.1, there is a module called lambda with a function lambda that uses more or less the same syntax. Instead of lambda x: x+1, you would instead write lambda(x, x+1). So the idea of including anonymous functions was around in Python circles before the syntax was locked in. I find that interesting. I also find it interesting that the common functional methods (all, any, map, filter) are basically built into Python core language. That is unusual for most imperative programming languages early-on. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using the Python Interpreter as a Reference
On Nov 28, 8:49 pm, Chris Angelico ros...@gmail.com wrote: On Tue, Nov 29, 2011 at 11:54 AM, DevPlayer devpla...@gmail.com wrote: To me, I would think the interpreter finding the coder's intended indent wouldn't be that hard. And just make the need for consistant spaces or tabs irrevelent simply by reformatting the indent as expected. Pretty much all my text editors can. The trouble with having a language declaration that a tab is equivalent to X spaces is that there's no consensus as to what X should be. Historically X has always been 8, and quite a few programs still assume this. I personally like 4. Some keep things narrow with 2. You can even go 1 - a strict substitution of \t with \x20. Once you declare it in your language, you immediately break everyone who uses anything different. ChrisA Yeah. We must remember the Unix users, espcially those who don't know how to hack Vim or bash. I've decided not to require a specific number of spaces. I am still teetering on whether to allow tabs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using the Python Interpreter as a Reference
On Nov 28, 5:57 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: On Mon, 28 Nov 2011 13:29:06 -0800, Travis Parks wrote: Exception handling is one of those subjects few understand and fewer can implement properly in modern code. Languages that don't support exceptions as part of their signature lead to capturing generic Exception all throughout code. It is one of those features I wish .NET had. At the same time, with my limited experience with Java, it has been a massive annoyance. Perhaps something to provide or just shut off via a command line parameter. What indications have there been that this has been a flaw? I can see it alienating a large group of up- and-coming developers. http://www.ibm.com/developerworks/java/library/j-jtp05254/index.html Note also that Bruce Eckel repeats a rumour that checked exceptions were *literally* an experiment snuck into the Java language while James Gosling was away on holiday. http://www.mindview.net/Etc/Discussions/UnCheckedExceptionComments Even if that is not true, checked exceptions are a feature that *in practice* seems to lead to poor exception handling and cruft needed only to satisfy the compiler: http://www.alittlemadness.com/2008/03/12/checked-exceptions-failed-ex... and other annoyances. It's main appeal, it seems to me, is to give a false sense of security to Java developers who fail to realise that under certain circumstances Java will raise certain checked exceptions *even if they are not declared*. E.g. null pointer exceptions. See also: http://java.dzone.com/articles/checked-exceptions-i-love-you and note especially the comment from the coder who says that he simply declares his functions to throw Exception (the most generic checked exception), thus defeating the whole point of checked exceptions. -- Steven I think all of the examples you listed referred specifically to most programmers finding ways around the annoyance. I have heard about throwing generic Exception or inheriting all custom exception types from RuntimeException. I did this quite often myself. In general, unchecked exceptions shouldn't be caught. They occur because of bad code and insufficient testing. Checked exceptions occur because of downed databases, missing files, network problems - things that may become available later without code changes. One day, I went through about 10,000 lines of code and moved argument checking code outside of try blocks because I realized I was handling some of them by accident. Here is the program: if me == idiot: exit(). People don't think about this, but the exceptions thrown by a module are part of that module's interface. Being able to make sure only what you expect to come out is important. Unlike Java, Unit requires you to opt in to using throws clauses. If you don't list one, one is generated for you automatically. The benefit: you can see what a function throws and protect yourself without all the babysitting. A lack of exception handling is big problem in .NET. I have had libraries from big names including Novell and Oracle throw NullReferenceExceptions because _they_ didn't know what would happen in cases where a DLL is missing or a dependency isn't installed. They could have done better testing, but if the biggest names in development can't manage to figure it, I say leave it up to the compiler. Returning nulls or special value in cases of failures takes us back to the days of C and Fortran. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using the Python Interpreter as a Reference
On Nov 26, 1:53 pm, Rick Johnson rantingrickjohn...@gmail.com wrote:
On Nov 20, 6:46 pm, Travis Parks jehugalea...@gmail.com wrote:
Hello:
I am currently working on designing a new programming language. It is
a compiled language, but I still want to use Python as a reference.
Python has a lot of similarities to my language, such as indentation
for code blocks,
I hope you meant to say *forced* indention for code blocks! Forced
being the key word here. What about tabs over spaces, have you decided
the worth of one over the other or are you going to repeat Guido's
folly?
And please, i love Python, but the language is a bit asymmetrical. Do
try to bring some symmetry to this new language. You can learn a lot
from GvR's triumphs, however, you can learn even more from his follys.
Personally, I find a lot of good things in Python. I thinking tabs are
out-of-date. Even the MAKE community wishes that the need for tabs
would go away and many implementations have done just that. I have
been seriously debating about whether to force a specific number of
spaces, such as the classic 4, but I am not sure yet. Some times, 2 or
even 8 spaces is appropriate (although I'm not sure when).
I have always found the standard library for Python to be disjoint.
That can be really beneficial where it keeps the learning curve down
and the size of the standard modules down. At the same time, it means
re-learning whenever you use a new module.
My language combines generators and collection initializers, instead
of creating a whole new syntax for comprehensions.
[| for i in 0..10: for j in 0.10: yield return i * j |]
Lambdas and functions are the same thing in my language, so no need
for a special keyword. I also distinguish between initialization and
assignment via the let keyword. Also, non-locals do not need to be
defined explicitly, since the scoping rules in Unit are far more
anal.
In reality though, it takes a certain level of arrogance to assume
that any language will turn out without bumps. It is like I was told
in college long ago, Only the smallest programs are bug free. I
think the same thing could be said for a language. The only language
without flaws would be so small that it would be useless.
I love these types of discussions though, because it helps me to be
aware. When designing a language, it is extremely helpful to hear what
language features have led to problems. For instance, C#'s foreach
loops internally reuse a variable, which translates to something like
this:
using (IEnumeratorT enumerator = enumerable.GetEnumerator())
{
T current;
while (enumerator.MoveNext())
{
current = enumerator.Current;
// inner loop code goes here
}
}
Since the same variable is reused, threads referencing the loop
variable work against whatever value is currently in the variable,
rather than the value when the thread was created. Most of the time,
this means every thread works against the same value, which isn't the
expected outcome. Moving the variable inside the loop _may_ help, but
it would probably be optimized back out of the loop by the compiler.
With the growth of threaded applications, these types of stack-based
optimizations may come to an end. That is why it is important for a
next-gen language to have a smarter stack - one that is context
sensitive. In Unit, the stack grows and shrinks like a dynamic array,
at each scope, rather than at the beginning and end of each function.
Sure, there's a slight cost in performance, but a boost in
consistency. If a programmer really wants the performance, they can
move the variable out of the loop themselves.
In fact, there are a lot of features in Unit that will come with
overhead, such as default arguments, non-locals, function-objects,
etc. However, the game plan is to avoid the overhead if it isn't used.
Some things, such as exception handling, will be hard to provide
without overhead. My belief is that, provided a tool, most developers
will use it and accept the slight runtime overhead.
I think everyone has an idea about what would make for the perfect
language. I am always willing to entertain ideas. I have pulled from
many sources: C#, Java, Python, JavaScript, F#, Lisp and more. The
hope is to provide as much expression with as much consistency as
possible. Just the other day I spent 2 hours trying to determine how
to create a null pointer (yeah, it took that long).
let pi = null as shared * Integer32 # null is always a pointer
Originally, I wanted 'as' to be a safe conversion. However, I decided
to make use of the 'try' keyword to mean a safe conversion.
let nd = try base as shared * Derived
let d = if nd.Succeeded: nd.Value else: null
# or, shorthand
let i = try Integer32.Parse(123) else 0
Of course, the last line could cost performance wise. For that reason,
Unit will allow for try versions of methods.
let Parse = public static method (value: String)
throws(FormatException UnderflowException OverflowException
Re: Using the Python Interpreter as a Reference
On Nov 22, 1:37 pm, Alan Meyer amey...@yahoo.com wrote: On 11/20/2011 7:46 PM, Travis Parks wrote: Hello: I am currently working on designing a new programming language. ... I have great respect for people who take on projects like this. Your chances of popularizing the language are small. There must be thousands of projects like this for every one that gets adopted by other people. However your chances of learning a great deal are large, including many things that you'll be able to apply to programs and projects that, at first glance, wouldn't appear to benefit from this kind of experience. If you get it working you'll have an impressive item to add to your resume. I suspect that you'll also have a lot of fun. Good luck with it. Alan I've been learning a lot and having tons of fun just designing the language. First, I get think about all of the language features that I find useful. Then I get to learn a little bit how they work internally. For instance, functions are first-class citizens in Unit, supporting closures. To make that happen meant wrapping such functions inside of types and silently elavating local variables to reference counted pointers. Or, I realized that in order to support default arguments, I would have to silently wrap parameters in types that were either set or not set. That way calls to the default command could simply be replaced by an if statement. It was a really subtle implementation detail. It is also fun thinking about what makes sense. For instance, Unit will support calling methods with named arguments. Originally, I thought about using the '=' operator: Foo(name=bob age=64) but, then I realized that the equals sign could be confused with assignment. Those types of syntactic conflicts occur quite often and lead to a lot of rethinking. Ultimately, somewhat good ideas get replaced with much better ideas. I had been contemplating Unit for months before the final look and feel of the language came into view. It isn't what I started out imagining, but I think it turned out better than I had originally planned. Recently, I rethought how functions looked, since the headers were too long: alias Predicate = functionT (value: readonly T) throws() returns(Boolean) let Any = public functionT (values: readonly IIterableT) (?predicate: PredicateT) throws() # ArgumentNullException inherits from UncheckedException returns(Boolean): # this can be on one line default predicate = (function value: true) assert predicate != null The predicate cannot be null. ArgumentNullException for value in values: if predicate(value): return true return false Most of the time, throws clauses, returns clauses and parameter type constraints can be left off. Plus, now they can all appear on one line. Assertions and default statements now appear in the body. Assertions now optionally take a message and the exception type to throw. So, yeah, this has been an awesome project so far. I have dozens of documents and I have been keeping up on a blog. I've even started implementing a simple recursive descent parser just to make sure the syntax doesn't conflict. Now it will be a matter of formally defining a grammer and implementing the backend of the compiler... which I've never done before. I have been thinking about compiling into a language like C++ or C instead of assembler for my first time through. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using the Python Interpreter as a Reference
On Nov 21, 12:44 am, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: On Mon, 21 Nov 2011 13:33:21 +1100, Chris Angelico wrote: What's your language's special feature? I like to keep track of languages using a slug - a simple one-sentence (or less) statement of when it's right to use this language above others. For example, Python is optimized for 'rapid deployment'. Python will save the world http://proyectojuanchacon.blogspot.com/2010/07/saving-world-with-pyth... -- Steven The language, psuedo name Unit, will be a low-level language capable of replacing C in most contexts. However, it will have integrated functional programming features (tail-end recursion optimization, tuples, currying, closures, function objects, etc.) and dynamic features (prototypical inheritance and late binding). It is a hybrid between C#, C++, F#, Python and JavaScript. The hope is that you won't pay for features you don't use, so it will run well on embedded devices as well as on desktops - that's to be seen. I'm no master compiler builder, here. The functional code is pretty basic: let multiply = function x y: return x * y # automatic generic arguments (integer here) let double = multiply _ 2 # short-hand currying - inlined if possible let doubled = [|0..10|].Apply(double) # double zero through 10 The dynamic code is pretty simple too: dynamic Prototype = function value: self.Value = value # simulated ctor Prototype.Double = function: self.Value * 2 # self refers to instance new Prototype(5).Double() # 10 new Prototype(6).Double() # 12 dynamic x = 5 # five wrapped with a bag x.Double = function: self * 2 x.Double() # 10 dynamic y = 6 y.Double = x.Double # member sharing y.Double() #12 The language also sports OOP features like are found in Java or C#: single inheritance; multiple interface inheritance; sealed, virtual and abstract types and members; explicit inheritance; extension methods and namespaces. The coolest feature will be its generics-oriented function signatures. By default everything is generic. You apply constraints to parameters, rather than specific types. For instance: let Average = function values: where values is ICountableInteger32 IIterableInteger32 assert values.Count 0 The values list cannot be empty. throws ArgumentException returns Float64 let sum = 0 for value in values: sum += value return sum / values.Count # floating point division As you can see, the function headers can be larger than the bodies themselves. They support type constraints, assertions (argument checking), exceptions enumeration, default parameters and return type information. All of them can be left out if the type of arguments can be inferred. This will not be an overnight project. :-) -- http://mail.python.org/mailman/listinfo/python-list
Using the Python Interpreter as a Reference
Hello: I am currently working on designing a new programming language. It is a compiled language, but I still want to use Python as a reference. Python has a lot of similarities to my language, such as indentation for code blocks, lambdas, non-locals and my language will partially support dynamic programming. Can anyone list a good introduction to the files found in the source code? I have been poking around the source code for a little bit and there is a lot there. So, I was hoping someone could point me to the good parts. I am also wondering whether some of the code was generated because I see state transition tables, which I doubt someone built by hand. Any help would be greatly appreciated. It will be cool to see how the interpreter works internally. I am still wonder whether designing the language (going on 4 months now) will be harder than implementing it. Thanks, Travis Parks -- http://mail.python.org/mailman/listinfo/python-list
Re: xmlrpclib date times and a trailing Z
On Nov 11, 7:20 pm, Travis Parks jehugalea...@gmail.com wrote: I am trying to connect to Marchex's a call tracking software using xmlrpclib. I was able to get some code working, but I ran into a problem dealing with transfering datetimes. When I construct a xmlrpclib.ServerProxy, I am setting the use_datetime flag to indicate that I want to automatically convert back and forth between date times in the datetime library. I have a working version that doesn't use this flag, and I have to convert from the xmlrpclib.DateTime type to the datetime.datetime type manually, via string parsing. The thing is, Marchex's API returns date's with a trailing Z, after the time component. I did some research and this is supposed to be an indicator that UTC was used. However, it doesn't seem like the xmlrpclib likes it very much. It looks like it is using this code internally: time.strptime(data, %Y %m%dT%H:%M:%S) This code doesn't look like it handles time zones at all. I guess, is there a way to tell xmlrpclib to include time zones when parsing date times? Thanks, Travis Parks I did some chatting on IRC and it seems that the date/time format is not very well defined in XML RPC specs. So, basically, Marchex is using a format that the XML RPC library doesn't support. Strangely, Marchex supports incoming dates with the format MMddThhmmss. It just spits dates back out with -MM-ddThh:mm:ssZ. The ISO8601 standard seems to be used a lot, so it is surprising the library doesn't try multiple formats, at least. I find it strange that the library, in light of the fact that date formats aren't standardized, doesn't provide the ability to configure this. I also find it strange that the library doesn't incorporate Basic Authentication using urllib2, but instead rolls its own method of putting username:password@ before the netloc. I wish Python's libraries acted more like an integrated framework than just unrelated libraries. I suppose I am spoiled from years of working with all-in-one frameworks managed by a single group. That is not the way C/C++ works or how Linux works. The power generated by using a conglomeration of unrelated libraries is indisputable, even if it can be a productivity killer and just plain confusing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
On Nov 8, 12:09 am, Lie Ryan lie.1...@gmail.com wrote: On 11/08/2011 01:21 PM, Travis Parks wrote: On Nov 7, 12:44 pm, John Gordongor...@panix.com wrote: Inj98tnf$qh...@reader1.panix.com John Gordongor...@panix.com writes: In415d875d-bc6d-4e69-bcf8-39754b450...@n18g2000vbv.googlegroups.com Travis Parksjehugalea...@gmail.com writes: Which web frameworks have people here used and which have they found to be: scalable, RAD compatible, performant, stable and/or providing good community support? I am really trying to get as much feedback as I've used Django and it seems to be a very nice framework. However I've only done one project so I haven't delved too deeply. You are probably looking for more detail than It's a nice framework :-) The database model in Django is powerful; it allows you to do queries in native Python code without delving into backend SQL stuff. I don't know how scalable/performant the database model is, as the one project I worked on didn't deal with a ton of data. (But I'd be surprised if it had poor performance.) The URL dispatcher provides a very nice and logical way to associate a given URL with a given method call. Community support is excellent. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies I started the battle today. The new guy was trying to sell me on CodeIgnitor. I haven't looked at it, but it is PHP, so I really want to avoid it. The good thing is that all of his friends have been telling him to get into Python. I have been trying to convince him that PHP isn't cut out for background services and is mostly a front- end language. Python is much more geared towards hardcore data processing. Why write the system in two languages? I have been spending a lot of time looking at the Pyramid project: the next generation of the Pylons project. It looks powerful, but it seems to be a lot more complex than Django. CodeIgniter is a very fine framework, however it builds on top of a shitty excuse of a language called PHP. I've found that Django has a much better debugging tools; when a Django page produces an exception, it would always produce a useful error page. I haven't been able to do the same in CodeIgniter (nor in any PHP framework I've used, I'm starting to think it's a language limitation); often when you have errors, PHP would just silently return empty or partial pages even with all the debugging flags on. IMO, Python has a much nicer choice of built-in data structure for data processing. Python has a much more mature object-orientation, e.g. I prefer writing l.append(x) rather than array_push(l, x). I think these qualities are what makes you think Python is much, much more suitable for data processing than PHP; and I wholesomely agree. Database abstraction-wise, Django's ORM wins hands down against CodeIgniter's ActiveRecord. CodeIgniter's ActiveRecord is basically just a thin wrapper that abstracts the perks of various database engine. Django's ORM is a full blown ORM, it handles foreign key relationships in OO way. The only disadvantage of Django's ORM is that since it's written in Python, if you need to write a program working on the same database that doesn't use Django nor Python, then you'll have a problem since you'll have to duplicate the foreign key relationships. With all the bashing of PHP, PHP do have a few advantages. PHP and CodeIgniter is much easier to set up and running than Django; and the ability to create a .php file and have it running without having to write the routing file is sometimes a bliss. And PHP are often used as their own templating language; in contrast with Django which uses a separate templating language. Having a full blown language as your templating language can be a double-edged sword, but it is useful nevertheless for experimental work. IMO, while it is easier to get up and running in PHP, in the long run Python is much better in almost any other aspects.- Hide quoted text - - Show quoted text - The good thing is that I got the new guy to convert his thinking towards Python. He did a little research of his own and realized what he was missing. He and I have been writing some tools in Python, accessing social networking sites, and have been pleasantly surprised by Python's rich support for web protocols. Even yesterday, I wrote some code using xmlrpclib and it blew the equivalent C# code out of the water. urllib2 and urlparse make it really easy to work against RESTful services. It seems like most of Google's APIs have a Python variant. We are thinking we will go along with Pyramid, rather than Django. It was a really hard decision to make. Django has a lot of community support and is integrated with PyDev in eclipse. Nonetheless, we are anticipating
xmlrpclib date times and a trailing Z
I am trying to connect to Marchex's a call tracking software using xmlrpclib. I was able to get some code working, but I ran into a problem dealing with transfering datetimes. When I construct a xmlrpclib.ServerProxy, I am setting the use_datetime flag to indicate that I want to automatically convert back and forth between date times in the datetime library. I have a working version that doesn't use this flag, and I have to convert from the xmlrpclib.DateTime type to the datetime.datetime type manually, via string parsing. The thing is, Marchex's API returns date's with a trailing Z, after the time component. I did some research and this is supposed to be an indicator that UTC was used. However, it doesn't seem like the xmlrpclib likes it very much. It looks like it is using this code internally: time.strptime(data, %Y %m%dT%H:%M:%S) This code doesn't look like it handles time zones at all. I guess, is there a way to tell xmlrpclib to include time zones when parsing date times? Thanks, Travis Parks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
On Nov 7, 12:44 pm, John Gordon gor...@panix.com wrote: In j98tnf$qh...@reader1.panix.com John Gordon gor...@panix.com writes: In 415d875d-bc6d-4e69-bcf8-39754b450...@n18g2000vbv.googlegroups.com Travis Parks jehugalea...@gmail.com writes: Which web frameworks have people here used and which have they found to be: scalable, RAD compatible, performant, stable and/or providing good community support? I am really trying to get as much feedback as I've used Django and it seems to be a very nice framework. However I've only done one project so I haven't delved too deeply. You are probably looking for more detail than It's a nice framework :-) The database model in Django is powerful; it allows you to do queries in native Python code without delving into backend SQL stuff. I don't know how scalable/performant the database model is, as the one project I worked on didn't deal with a ton of data. (But I'd be surprised if it had poor performance.) The URL dispatcher provides a very nice and logical way to associate a given URL with a given method call. Community support is excellent. -- John Gordon A is for Amy, who fell down the stairs gor...@panix.com B is for Basil, assaulted by bears -- Edward Gorey, The Gashlycrumb Tinies I started the battle today. The new guy was trying to sell me on CodeIgnitor. I haven't looked at it, but it is PHP, so I really want to avoid it. The good thing is that all of his friends have been telling him to get into Python. I have been trying to convince him that PHP isn't cut out for background services and is mostly a front- end language. Python is much more geared towards hardcore data processing. Why write the system in two languages? I have been spending a lot of time looking at the Pyramid project: the next generation of the Pylons project. It looks powerful, but it seems to be a lot more complex than Django. -- http://mail.python.org/mailman/listinfo/python-list
Python ORMs Supporting POPOs and Substituting Layers in Django
Hello: A new guy showed up at work a few weeks ago and has started talking about replacing a 6 month old project, written in ASP.NET MVC, with an open source solution that can handle massive scaling. I think his primary concern is the potential need for massive web farms in the future. In order to prevent high licensing costs, I think he wants to move everything to open source technologies, such as the LAMP stack. I also don't think he truly understands what ASP.NET MVC is and thinks it is the older WebForms. I have been researching open source MVC frameworks and came across Django. It looks like an awesome tool, but I am willing to look at others. I have experience in Python (and enough in PHP to want to avoid it and absolutely none in Ruby) so I think it would be a good language to develop in. I was wondering if there were any ORMs for Python that used POPOs (plain old Python objects). There is a lot of business logic in my system, and so I want to keep my data objects simple and stupid. I want the ORM to be responsible for detecting changes to objects after I send them back to the data layer (rather than during business layer execution). Additionally, being a stateless environment, tracking objects' states isn't very useful anyway. Honestly, I doubt this guy is going to get his wish. The people paying for the application aren't going to be willing to throw 6 months of work down the drain. Never the less, I want to have plenty of research under my belt before being asked what my thoughts are. He was talking about using the Zend Framework with PHP, but I want to avoid that if possible. Django seems like one of the best MVC solutions in the Python arena. I would be willing to replace Django's ORM solution with something else, especially if it supported POPOs. I could even map all of the non-POPOs to POPOs if I needed to, I guess. Finally, I wanted to ask whether anyone has tried having Django call out to Python 3 routines. I am okay using Python 2.7 in Django, if I can have the controllers call business logic implemented in Python 3, accepting POPOs from the data layer. Django would really just be a coordinator: grab data from Django ORM, convert results into POPOs, load up Python 3 module with business logic, passing POPOs, returning POPOs and then converting those to view models. I'm sweating just thinking about it. My guess is that there would be a severe penalty for crossing process boundaries... but any insights would be appreciated. Thanks, Travis Parks -- http://mail.python.org/mailman/listinfo/python-list
Re: Python ORMs Supporting POPOs and Substituting Layers in Django
On Nov 5, 4:11 pm, Travis Parks jehugalea...@gmail.com wrote: Hello: A new guy showed up at work a few weeks ago and has started talking about replacing a 6 month old project, written in ASP.NET MVC, with an open source solution that can handle massive scaling. I think his primary concern is the potential need for massive web farms in the future. In order to prevent high licensing costs, I think he wants to move everything to open source technologies, such as the LAMP stack. I also don't think he truly understands what ASP.NET MVC is and thinks it is the older WebForms. I have been researching open source MVC frameworks and came across Django. It looks like an awesome tool, but I am willing to look at others. I have experience in Python (and enough in PHP to want to avoid it and absolutely none in Ruby) so I think it would be a good language to develop in. I was wondering if there were any ORMs for Python that used POPOs (plain old Python objects). There is a lot of business logic in my system, and so I want to keep my data objects simple and stupid. I want the ORM to be responsible for detecting changes to objects after I send them back to the data layer (rather than during business layer execution). Additionally, being a stateless environment, tracking objects' states isn't very useful anyway. Honestly, I doubt this guy is going to get his wish. The people paying for the application aren't going to be willing to throw 6 months of work down the drain. Never the less, I want to have plenty of research under my belt before being asked what my thoughts are. He was talking about using the Zend Framework with PHP, but I want to avoid that if possible. Django seems like one of the best MVC solutions in the Python arena. I would be willing to replace Django's ORM solution with something else, especially if it supported POPOs. I could even map all of the non-POPOs to POPOs if I needed to, I guess. Finally, I wanted to ask whether anyone has tried having Django call out to Python 3 routines. I am okay using Python 2.7 in Django, if I can have the controllers call business logic implemented in Python 3, accepting POPOs from the data layer. Django would really just be a coordinator: grab data from Django ORM, convert results into POPOs, load up Python 3 module with business logic, passing POPOs, returning POPOs and then converting those to view models. I'm sweating just thinking about it. My guess is that there would be a severe penalty for crossing process boundaries... but any insights would be appreciated. Thanks, Travis Parks Which web frameworks have people here used and which have they found to be: scalable, RAD compatible, performant, stable and/or providing good community support? I am really trying to get as much feedback as I can, to help form an unbiased opinion in case I need to make a recommendation... or fight an all out battle. -- http://mail.python.org/mailman/listinfo/python-list
Re: Algorithms Library - Asking for Pointers
On Sep 3, 12:35 am, Chris Torek nos...@torek.net wrote: In article 18fe4afd-569b-4580-a629-50f6c7482...@c29g2000yqd.googlegroups.com Travis Parks jehugalea...@gmail.com wrote: [Someone] commented that the itertools algorithms will perform faster than the hand-written ones. Are these algorithms optimized internally? They are written in C, so avoid a lot of CPython interpreter overhead. Mileage in Jython, etc., may vary... -- In-Real-Life: Chris Torek, Wind River Systems Intel require I note that my opinions are not those of WRS or Intel Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603 email: gmail (figure it out) http://web.torek.net/torek/index.html I thought I would point out that many of the itertools functions change between 2.x and 3.x versions. Since 2.7 is supposed to be the last 2.x language, I suppose I will wait until 3.2 becomes the norm before I incorporate some of these changes. In the mean time, I will starting working on algorithms that work against Sequences. I think a really important lesson is that Python really doesn't need an algorithms library, like many others do. A lot of the common algorithms are supported by the syntax itself. All my library did was allow for easier function composition. -- http://mail.python.org/mailman/listinfo/python-list
Algorithms Library - Asking for Pointers
Hello: I am working on an algorithms library. It provides LINQ like functionality to Python iterators. Eventually, I plan on having feaures that work against sequences and mappings. I have the code up at http://code.google.com/p/py-compass. This is my first project in Python, so I'd like some feedback. I want to know if I am following conventions (overall style and quality of code). Thanks, Travis Parks -- http://mail.python.org/mailman/listinfo/python-list
Re: Handling 2.7 and 3.0 Versions of Dict
On Sep 2, 12:36 pm, Gabriel Genellina gagsl-...@yahoo.com.ar wrote: En Wed, 31 Aug 2011 22:28:09 -0300, Travis Parks jehugalea...@gmail.com escribi : On Aug 31, 7:37 pm, Gregory Ewing greg.ew...@canterbury.ac.nz wrote: Ian Kelly wrote: if sys.version_info (3,): getDictValues = dict.itervalues else: getDictValues = dict.values (which is basically what the OP was doing in the first place). My problem was that I didn't understand the scoping rules. It is still strange to me that the getValues variable is still in scope outside the if/else branches. Those if/else are at global scope. An 'if' statement does not introduce a new scope; so getDictValues, despite being indented, is defined at global scope, and may be used anywhere in the module. -- Gabriel Genellina Does that mean the rules would be different inside a function? -- http://mail.python.org/mailman/listinfo/python-list
Re: Algorithms Library - Asking for Pointers
On Sep 2, 4:09 pm, Ian Kelly ian.g.ke...@gmail.com wrote: On Fri, Sep 2, 2011 at 10:59 AM, Travis Parks jehugalea...@gmail.com wrote: Hello: I am working on an algorithms library. It provides LINQ like functionality to Python iterators. Eventually, I plan on having feaures that work against sequences and mappings. I have the code up athttp://code.google.com/p/py-compass. This is my first project in Python, so I'd like some feedback. I want to know if I am following conventions (overall style and quality of code). Sure, here are my comments. In the forever and __forever functions, your use of the term generator is confusing. __forever is a generator function, because it has a yield statement. Its argument, called generator, appears to be a callable, not a generator or even necessarily a generator function. Also, note that __forever(lambda: value) is functionally equivalent to the more efficient itertools.repeat(value). The staticmethod __next(iterator) accesses the class it is defined in, which suggests that it might be better written as a classmethod __next(cls, iterator). Each of the LINQ-style methods is divided into two parts: the public method that contains the docstring and some argument checks, and a private staticmethod that contains the implementation. I'm not certain what the purpose of that is. If it's to facilitate overriding the implementation in subclasses, then you need to change the names of the private methods to start with only one _ character so that they won't be mangled by the compiler. The comments before each method that only contain the name of the immediately following method are redundant. aggregate: the default aggregator is unintuitive to me. I would make it a required field and add a separate method called sum that calls aggregate with the operator.add aggregator. Also, the implementation doesn't look correct. Instead of passing in each item to the aggregator, you're passing in the number of items seen so far? The LINQ Aggregate method is basically reduce, so rather than reinvent the wheel I would suggest this: # MISSING is a unique object solely defined to represent missing arguments. # Unlike None we can safely assume it will never be passed as actual data. MISSING = object() def aggregate(self, aggregator, seed=MISSING): if seed is self.MISSING: return reduce(aggregator, self._iterable) else: return reduce(aggregator, self._iterable, seed) Note for compatibility that in Python 3 the reduce function has been demoted from a builtin to a member of the functools module. any: the name of this method could cause some confusion with the any builtin that does something a bit different. compare: the loop would more DRY as a for loop: def __compare(first, second, comparison): for firstval, secondval in itertools.izip_longest(first, second, fillvalue=self.MISSING): if firstval is self.MISSING: return -1 elif secondval is self.MISSING: return 1 else: result = comparison(firstval, secondval) if result != 0: return result return 0 concatenate: again, no need to reinvent the wheel. This should be more efficient: def concatenate(self, other): return extend(itertools.chain(self.__iterable, other)) equals: could be just return self.compare(other, comparison) == 0 __last: the loop could be a for loop: # assume we're looking at the last item and try moving to the next item = result.Value for item in iterator: pass return item lastOrDefault: there's a lot of repeated logic here. This could just be: def lastOrDefault(self, default=None): try: return self.last() except ValueError: return default map / forEach: .NET has to separate these into separate methods due to static typing. It seems a bit silly to have both of them in Python. Also, map would be more efficient as return itertools.imap(mapper, self.__iterable) max / min: it would be more efficient to use the builtin: def max(self, key): return max(self.__iterable, key=key) If somebody really needs to pass a comparison function instead of a key function, they can use functools.cmp_to_key. randomSamples: a more canonical way to pass the RNG would be to pass an instance of random.Random rather than an arbitrary function. Then to get a random integer you can just call generator.randrange(total). Note that for the default you can just use the random module itself, which provides default implementations of all the Random methods. Also, for loops: def __randomSamples(iterable, sampleCount, generator): samples = [] iterator = iter(iterable) # fill up the samples with the items from the beginning of the iterable for i, value in zip(xrange(sampleCount), iterator): samples.append(value
Re: Algorithms Library - Asking for Pointers
On Sep 2, 6:49 pm, Travis Parks jehugalea...@gmail.com wrote: On Sep 2, 4:09 pm, Ian Kelly ian.g.ke...@gmail.com wrote: On Fri, Sep 2, 2011 at 10:59 AM, Travis Parks jehugalea...@gmail.com wrote: Hello: I am working on an algorithms library. It provides LINQ like functionality to Python iterators. Eventually, I plan on having feaures that work against sequences and mappings. I have the code up athttp://code.google.com/p/py-compass. This is my first project in Python, so I'd like some feedback. I want to know if I am following conventions (overall style and quality of code). Sure, here are my comments. In the forever and __forever functions, your use of the term generator is confusing. __forever is a generator function, because it has a yield statement. Its argument, called generator, appears to be a callable, not a generator or even necessarily a generator function. Also, note that __forever(lambda: value) is functionally equivalent to the more efficient itertools.repeat(value). The staticmethod __next(iterator) accesses the class it is defined in, which suggests that it might be better written as a classmethod __next(cls, iterator). Each of the LINQ-style methods is divided into two parts: the public method that contains the docstring and some argument checks, and a private staticmethod that contains the implementation. I'm not certain what the purpose of that is. If it's to facilitate overriding the implementation in subclasses, then you need to change the names of the private methods to start with only one _ character so that they won't be mangled by the compiler. The comments before each method that only contain the name of the immediately following method are redundant. aggregate: the default aggregator is unintuitive to me. I would make it a required field and add a separate method called sum that calls aggregate with the operator.add aggregator. Also, the implementation doesn't look correct. Instead of passing in each item to the aggregator, you're passing in the number of items seen so far? The LINQ Aggregate method is basically reduce, so rather than reinvent the wheel I would suggest this: # MISSING is a unique object solely defined to represent missing arguments. # Unlike None we can safely assume it will never be passed as actual data. MISSING = object() def aggregate(self, aggregator, seed=MISSING): if seed is self.MISSING: return reduce(aggregator, self._iterable) else: return reduce(aggregator, self._iterable, seed) Note for compatibility that in Python 3 the reduce function has been demoted from a builtin to a member of the functools module. any: the name of this method could cause some confusion with the any builtin that does something a bit different. compare: the loop would more DRY as a for loop: def __compare(first, second, comparison): for firstval, secondval in itertools.izip_longest(first, second, fillvalue=self.MISSING): if firstval is self.MISSING: return -1 elif secondval is self.MISSING: return 1 else: result = comparison(firstval, secondval) if result != 0: return result return 0 concatenate: again, no need to reinvent the wheel. This should be more efficient: def concatenate(self, other): return extend(itertools.chain(self.__iterable, other)) equals: could be just return self.compare(other, comparison) == 0 __last: the loop could be a for loop: # assume we're looking at the last item and try moving to the next item = result.Value for item in iterator: pass return item lastOrDefault: there's a lot of repeated logic here. This could just be: def lastOrDefault(self, default=None): try: return self.last() except ValueError: return default map / forEach: .NET has to separate these into separate methods due to static typing. It seems a bit silly to have both of them in Python. Also, map would be more efficient as return itertools.imap(mapper, self.__iterable) max / min: it would be more efficient to use the builtin: def max(self, key): return max(self.__iterable, key=key) If somebody really needs to pass a comparison function instead of a key function, they can use functools.cmp_to_key. randomSamples: a more canonical way to pass the RNG would be to pass an instance of random.Random rather than an arbitrary function. Then to get a random integer you can just call generator.randrange(total). Note that for the default you can just use the random module itself, which provides default implementations of all the Random methods. Also, for loops: def __randomSamples(iterable, sampleCount, generator): samples = [] iterator = iter(iterable
Closures and Partial Function Application
I was a little disappointed the other day when I realized that closures were read-only. I like to use closures quite a bit. Can someone explain why this limitation exists? Secondly, since I can cheat by wrapping the thing being closure-ified, how can I write a simple wrapper that has all the same members as the thing (decorator), that then applies them to the underlying thing? I also like partial function application. What is the easiest way of achieving this in Python? Would it look something like this: def foo(x, y): return x + y xFoo = lambda y: foo(10, y) -- http://mail.python.org/mailman/listinfo/python-list
Re: Closures and Partial Function Application
On Aug 31, 1:18 pm, Chris Rebert c...@rebertia.com wrote: On Wed, Aug 31, 2011 at 9:45 AM, Travis Parks jehugalea...@gmail.com wrote: I was a little disappointed the other day when I realized that closures were read-only. I like to use closures quite a bit. Assuming I'm intuiting your question correctly, then you're incorrect; they are read/write. You just need a `nonlocal` declaration for the variables in question. Seehttp://www.python.org/dev/peps/pep-3104/ andhttp://docs.python.org/release/3.1.3/reference/simple_stmts.html#nonl... for details. Cheers, Chris Cool. So I just need to put nonlocal in front of the variable name. -- http://mail.python.org/mailman/listinfo/python-list
Re: Closures and Partial Function Application
On Aug 31, 1:51 pm, Travis Parks jehugalea...@gmail.com wrote: On Aug 31, 1:18 pm, Chris Rebert c...@rebertia.com wrote: On Wed, Aug 31, 2011 at 9:45 AM, Travis Parks jehugalea...@gmail.com wrote: I was a little disappointed the other day when I realized that closures were read-only. I like to use closures quite a bit. Assuming I'm intuiting your question correctly, then you're incorrect; they are read/write. You just need a `nonlocal` declaration for the variables in question. Seehttp://www.python.org/dev/peps/pep-3104/ andhttp://docs.python.org/release/3.1.3/reference/simple_stmts.html#nonl... for details. Cheers, Chris Cool. So I just need to put nonlocal in front of the variable name. Am I doing something wrong, here? nonlocal isn't registering. Which version did this get incorporated? -- http://mail.python.org/mailman/listinfo/python-list
Re: Closures and Partial Function Application
On Aug 31, 2:18 pm, Ian Kelly ian.g.ke...@gmail.com wrote: On Wed, Aug 31, 2011 at 12:02 PM, Travis Parks jehugalea...@gmail.com wrote: Am I doing something wrong, here? nonlocal isn't registering. Which version did this get incorporated? 3.0 Ah, okay. It would be really useful for unit testing. Unfortunately, I want to make the code I am writing compatible with 2.x and 3.x. I will just deal with it until 3.x takes over. Glad to know Guido sees the importance. -- http://mail.python.org/mailman/listinfo/python-list
Re: Closures and Partial Function Application
On Aug 31, 2:03 pm, bruno.desthuilli...@gmail.com bruno.desthuilli...@gmail.com wrote: On 31 août, 18:45, Travis Parks jehugalea...@gmail.com wrote: I was a little disappointed the other day when I realized that closures were read-only. I like to use closures quite a bit. They are not _strictly_ read only, but Python being first and foremost an OO language, it's usually way simpler to use OO instead of closures when you start needing such features. I like to leave OO to large-scale architectures and leave functional paradigms for implementation details. Writing an entire class for wrapping an int seems excessive. Especially if that code is limited to a small scope. I agree, though, that there is a time and a place for everything. -- http://mail.python.org/mailman/listinfo/python-list
Re: Handling 2.7 and 3.0 Versions of Dict
On Aug 31, 7:37 pm, Gregory Ewing greg.ew...@canterbury.ac.nz wrote:
Ian Kelly wrote:
if sys.version_info (3,):
getDictValues = dict.itervalues
else:
getDictValues = dict.values
(which is basically what the OP was doing in the first place).
And which he seemed to think didn't work for some
reason, but it seems fine as far as I can tell:
Python 2.7 (r27:82500, Oct 15 2010, 21:14:33)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type help, copyright, credits or license for more information.
gv = dict.itervalues
d = {1:'a', 2:'b'}
gv(d)
dictionary-valueiterator object at 0x2aa210
% python3.1
Python 3.1.2 (r312:79147, Mar 2 2011, 17:43:12)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type help, copyright, credits or license for more information.
gv = dict.values
d = {1:'a', 2:'b'}
gv(d)
dict_values(['a', 'b'])
--
Greg
My problem was that I didn't understand the scoping rules. It is still
strange to me that the getValues variable is still in scope outside
the if/else branches.
--
http://mail.python.org/mailman/listinfo/python-list
Handling 2.7 and 3.0 Versions of Dict
I am writing a simple algorithms library that I want to work for both
Python 2.7 and 3.x. I am writing some functions like distinct, which
work with dictionaries under the hood. The problem I ran into is that
I am calling itervalues or values depending on which version of the
language I am working in. Here is the code I wrote to overcome it:
import sys
def getDictValuesFoo():
if sys.version_info (3,):
return dict.itervalues
else:
return dict.values
getValues = getDictValuesFoo()
def distinct(iterable, keySelector = (lambda x: x)):
lookup = {}
for item in iterable:
key = keySelector(item)
if key not in lookup:
lookup[key] = item
return getValues(lookup)
I was surprised to learn that getValues CANNOT be called as if it were
a member of dict. I figured it was more efficient to determine what
getValues was once rather than every time it was needed.
First, how can I make the method getValues private _and_ so it only
gets evaluated once? Secondly, will the body of the distinct method be
evaluated immediately? How can I delay building the dict until the
first value is requested?
I noticed that hashing is a lot different in Python than it is in .NET
languages. .NET supports custom equality comparers that can override
a type's Equals and GetHashCode functions. This is nice when you can't
change the class you are hashing. That is why I am using a key
selector in my code, here. Is there a better way of overriding the
default hashing of a type without actually modifying its definition? I
figured a requesting a key was the easiest way.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Checking Signature of Function Parameter
On Aug 29, 2:30 am, Nobody nob...@nowhere.com wrote: On Sun, 28 Aug 2011 14:20:11 -0700, Travis Parks wrote: More importantly, I want to make sure that predicate is callable, accepting a thing, returning a bool. The callable part is do-able, the rest isn't. The predicate may accept an arbitrary set of arguments via the *args and/or **kwargs syntax, and pass these on to some other function. Exactly *which* function may be the result of an arbitrarily complex expression. Or it may not even call another function, but just use the arbitrary set of arguments in an arbitrarily complex manner. IOW, determining in advance what will or won't work is actually impossible. Thanks for everyone's input. I decided that I will put some basic checks up front, like is it None, is it Iterable and is it callable. Other than that, I am letting things slide. Asking for forgiveness is always easier anyway. Just so everyone knows, I am defining these methods inside a class called IterableExtender: class IterableExtender(collections.Iterable):... I wanted to allow for calls like this: extend(range(0, 1000)).map(lambda x: x * x).where(lambda x: x % 2 == 0).first(lambda x: x % 7 == 0) It allows me to compose method calls similarly to LINQ in C#. I think this looks better than: first(where(map(range(0, 1000), lambda x: x * x, lambda x: x % 2 == 0, lambda x : x % 7 == 0))) Internally to the class, there are private static methods taking raw inputs and performing no checks. The public instance methods are responsible for checking input arguments and wrapping results. Eventually, I will start working on algorithms that work on MutableSequences, but for now I am just doing Iterables. This is turning out to be a great learning experience. -- http://mail.python.org/mailman/listinfo/python-list
Re: Checking Signature of Function Parameter
On Aug 29, 1:42 pm, Ian Kelly ian.g.ke...@gmail.com wrote: On Mon, Aug 29, 2011 at 10:45 AM, Travis Parks jehugalea...@gmail.com wrote: I wanted to allow for calls like this: extend(range(0, 1000)).map(lambda x: x * x).where(lambda x: x % 2 == 0).first(lambda x: x % 7 == 0) It allows me to compose method calls similarly to LINQ in C#. I think this looks better than: first(where(map(range(0, 1000), lambda x: x * x, lambda x: x % 2 == 0, lambda x : x % 7 == 0))) FWIW, I would be inclined to write that in Python like this: def first(iterable): try: return next(iter(iterable)) except StopIteration: raise ValueError(iterable was empty) squares = (x * x for x in range(0, 1000)) first(x for x in squares if x % 14 == 0) Python's comprehensions make the need for many of the methods I am writing unnecessary. Which probably explains why no ones really bothered to write one before. The only problem I have above is either the composition causes complex method calls first(where(map(range(..., it requires complex comprehensions or it requires breaking the code into steps. Even my approach has problems, such as the overhead of carrying an invisible wrapper around. It does a bit too much to comfortably be a one-liner no matter which way you write it, so I split it into two. Cheers, Ian Yeah. I have already seen a lot of better ways of writing my code based solely on your example. I didn't know about iter as a built-in function. I have been calling __iter__ directly. I also need to think more about whether methods like where and map are going to be beneficial. The good thing is that someone will be able to use my wrapper in any context where an Iterable can be used. It will allow someone to switch between styles on the fly. I'm still not convinced that this library is going to be very pythony. I wrote a post a few days ago about how I know the syntax and libraries fairly well, but I don't have the philosophy. I haven't seen a lot of tricks and I am never sure what is the norm in Python. I am sure if an experienced Python programmer looked at my code, they'd immediately know I was missing a few things. -- http://mail.python.org/mailman/listinfo/python-list
Checking Signature of Function Parameter
I am trying to write an algorithms library in Python. Most of the functions will accept functions as parameters. For instance, there is a function called any: def any(source, predicate): for item in source: if predicate(item): return true; return false; There are some things I want to make sure of. 1) I want to make sure that source is iterable. 2) More importantly, I want to make sure that predicate is callable, accepting a thing, returning a bool. This is what I have so far: if source is None: raise ValueError() if not isinstanceof(source, collections.iterable): raise TypeError() if not callable(predicate): raise TypeError() The idea here is to check for issues up front. In some of the algorithms, I will be using iterators, so bad arguments might not result in a runtime error until long after the calls are made. For instance, I might implement a filter method like this: def where(source, predicate): for item in source: if predicate(item): yield item Here, an error will be delayed until the first item is pulled from the source. Of course, I realize that functions don't really have return types. Good thing is that virtually everything evaluates to a boolean. I am more concerned with the number of parameters. Finally, can I use decorators to automatically perform these checks, instead of hogging the top of all my methods? -- http://mail.python.org/mailman/listinfo/python-list
Re: Checking Signature of Function Parameter
On Aug 28, 5:31 pm, Chris Angelico ros...@gmail.com wrote: On Mon, Aug 29, 2011 at 7:20 AM, Travis Parks jehugalea...@gmail.com wrote: if source is None: raise ValueError() if not isinstanceof(source, collections.iterable): raise TypeError() if not callable(predicate): raise TypeError() Easier: Just ignore the possibilities of failure and carry on with your code. If the source isn't iterable, you'll get an error raised by the for loop. If the predicate's not callable, you'll get an error raised when you try to call it. The only consideration you might need to deal with is that the predicate's not callable, and only if you're worried that consuming something from your source would be a problem (which it won't be with the normal iterables - strings, lists, etc, etc). Otherwise, just let the exceptions be raised! ChrisA I guess my concern is mostly with the delayed exceptions. It is hard to find the source of an error when it doesn't happen immediately. I am writing this library so all of the calls can be chained together (composed). If this nesting gets really deep, finding the source is hard to do, even with a good debugger. Maybe I should give up on it, like you said. I am still familiarizing myself with the paradigm. I want to make sure I am developing code that is consistent with the industry standards. -- http://mail.python.org/mailman/listinfo/python-list
Mastering Python... Best Resources?
I know the Python syntax pretty well. I know a lot of the libraries and tools. When I see professional Python programmer's code, I am often blown away with the code. I realized that even though I know the language, I know nothing about using it effectively. I would like to start using Python more in my professional career. Where can I find resources that will take my skills to the next level? I would prefer to watch a streaming video series, if possible. I've read quite a few books about Python. They cover a lot of topics, but none of them covered common conventions or hacks. I mean, I got good at C++ reading books by Scott Meyers, who concentrated on common idioms, things to avoid, the proper way to do things, etc. Right now, I am at that point where I know how to do write just about anything in the language. However, I still have that hesitation I get when I'm just not sure what's the right way. -- http://mail.python.org/mailman/listinfo/python-list
Re: Mastering Python... Best Resources?
On Aug 26, 8:44 am, Chris Angelico ros...@gmail.com wrote: On Fri, Aug 26, 2011 at 10:33 PM, Travis Parks jehugalea...@gmail.com wrote: I know the Python syntax pretty well. I know a lot of the libraries and tools. When I see professional Python programmer's code, I am often blown away with the code. I realized that even though I know the language, I know nothing about using it effectively. I would say that there are three aspects to using Python effectively: 1) Understanding the syntax, which you've mastered. 2) Understanding the philosophy 3) Knowing algorithms. The second is more or less what you're asking for, but the language-independent third may be more useful to you. This is correct Python syntax (#1), and decently Pythonic style (#2), but a hopelessly flawed algorithm (#3): def fib(x): return fib(x-1) + fib(x-2) if x2 else 1 Or: def fib(x): if x3: return 1 return fib(x-1) + fib(x-2) Both versions are clean and easy to read, but neither would be what I'd call brilliant code. You can get books on algorithms from all sorts of places, and with a very few exceptions, everything you learn with apply to Python and also to every other language you use. ChrisA Well, I think I am going more for #2. I know about things like data structures and algorithms... in your case memoization. Here is a good example of what I am talking about. Someone took the time to write quicksort in a single line of code: def qsortr(list): return [] if list==[] else qsortr([x for x in list[1:] if x list[0]]) + [list[0]] + qsortr([x for x in list[1:] if x = list[0]]) I would never even think to use list comprehensions and splicing like that. I would write this code the same way I'd write it in C++/C#. I'm aware that writing code like the above example is probably bad practice (and that the implementation here has some major inefficiencies), but it is the mentality that goes into it. I haven't gotten to the point where I can truly use the language features to my full advantage. I haven't seen enough tricks to be effective. I feel like there is so much of the language I am not utilizing because I'm still thinking in terms of a less powerful language. I was hoping to find a series that would familiarize me with how real Python programmers get things done. -- http://mail.python.org/mailman/listinfo/python-list
Re: Mastering Python... Best Resources?
On Aug 26, 9:28 am, Chris Angelico ros...@gmail.com wrote: On Fri, Aug 26, 2011 at 10:58 PM, Travis Parks jehugalea...@gmail.com wrote: I haven't gotten to the point where I can truly use the language features to my full advantage. I haven't seen enough tricks to be effective. I feel like there is so much of the language I am not utilizing because I'm still thinking in terms of a less powerful language. I was hoping to find a series that would familiarize me with how real Python programmers get things done. Ah! Then I recommend poking around with the standard library. No guarantees that it's ALL good code, but it probably will be. In any case, it sounds like you're well able to evaluate code in your own head and recognize the good from the ugly. In the source distribution (I'm looking at the latest straight from hg, but presumably it's the same everywhere), there's a whole lot of .py files in ./Lib - there's sure to be some good examples in there somewhere. ChrisA I've been thinking about going through the docs on the main website. Cool thing is it has links to the actual lib files. I was checking out string.py yesterday. I was searching all over youtube for good videos of some type. Google has an intro course, but it didn't really do much for me. Microsoft has these series called 'Going Deep' that occasionally runs something super in-depth. The videos on C++ and the STL are really excellent. I was hoping someone had taken the time to create a similar series for Python. I can't help but remember my one professor in college, who really made pointers, bitwise arithmetic and low level OS operations make sense. He explained to us a lot about how the STL worked and showed us tons of C++/STL hacks. I probably learned more in the 2 years I had classes with him than I have in all the time I've programmed. To get that type of insight into another language, like Python, would be the ultimate gift for someone like me. Personally, I am tired of working in languages that don't strongly support functional paradigms. -- http://mail.python.org/mailman/listinfo/python-list
Re: Mastering Python... Best Resources?
On Aug 26, 11:12 am, Roy Smith r...@panix.com wrote:
In article
2309ec4b-e9a3-4330-9983-1c621ac16...@ea4g2000vbb.googlegroups.com,
Travis Parks jehugalea...@gmail.com wrote:
I know the Python syntax pretty well. I know a lot of the libraries
and tools. When I see professional Python programmer's code, I am
often blown away with the code. I realized that even though I know the
language, I know nothing about using it effectively.
In a sense, I'm in the same boat as you. I've been using Python since
before the 2.0 series, and I tend to think of the language in much the
same way as I did back then. Which is to say I don't use the language,
as it currently exists, as effectively as I might.
Here's some things I suggest you look at:
Iterators. This is such a powerful concept. When I started with the
language, iterators largely meant the difference between range() and
xrange(). Now we've got a whole ecosystem which has grown up around
them (x + comprehension for x in {'list', 'dictionary', 'set'}), not
to mention generators and generator expressions. And the itertools
library.
Decorators. Another powerful concept. We use these in our web servers
for all sorts of cool things. Adding cacheing. Imposing prerequisites
on route calls. I still don't think of using these immediately, but I
do see the notational convenience they provide for many things.
Context Managers. One of the (very few) things that I always found
lacking in Python compared to C++ was deterministic object destruction.
Context managers give you this. I'm still exploring all the neat things
you can do with them.
The full range of containers. I started with lists, tuples, and
dictionaries. Now we've got sets, frozensets, named tuples, deques,
Counters, defaultdicts (I love those), heaps, and I'm sure a few others
I've missed. List and dicts are such well designed containers, you can
do almost anything with just those two, but all the other new ones often
make things quicker, simpler, and more obvious.
The profiler. Most people obsess about performance early on and don't
realize that most of their guesses about what's fast and what's slow are
probably wrong. Learn to use the profiler and understand what it's
telling you.
Unittest. Testing is, in general, a neglected practice in most software
development shops, and that's a shame. Python has some really good
capabilities to support testing which you should get familiar with.
Unittest is just one of them. There's also doctest, nose, and a bunch
of other contributed modules. Look at them all, learn at least one of
them well, and use it for everything you write.
I've read quite a few books about Python. They cover a lot of topics,
but none of them covered common conventions or hacks. I mean, I got
good at C++ reading books by Scott Meyers, who concentrated on common
idioms, things to avoid, the proper way to do things, etc.
Ugh. The problem with Meyers's books is that they are needed in the
first place. C++ is such a horribly complicated language, you really
can't use it without making a serious study of it. There's too many
gotchas that you MUST know to avoid disaster with even the most basic
programs.
Python isn't that way. You can learn a small, basic subset of the
language and get a lot done. You may not be doing things the most
effective way, but you're also not going to be looking at memory
corruption because you didn't understand the details of object lifetimes
or how type promotion, function overloading, and implicit temporary
object construction all interact.
Thanks for the input.
I had been writing my Compass project (http://compass.codeplex.com) in
Pythonese. I was planning on implementing a lot of the features of MS'
LINQ in Python iterators, too. I am surprised that there aren't a ton
of Python libraries for general purpose algorithms. yield is one of
my favorite keywords. :-)
I will take a look at decorators especially. I see them being used for
properties and other coolness. I started playing with unittest the
other day.
unittest.main(exit=False) -- took me a while to find
I will look at the containers, too. I have been trying to push tuple
syntax support in C# for years now. Named tuples are so useful.
I agree that C++ is too complicated. Bjarne should have cared less
about backward compatibility with C and fixed some of the issues with
it. He should have also made some of the defaults more intuitive -
like ctor initializers being reorganized to the same order as the
backing fields... that'll getcha. Function argument order evaluation.
Oh no! It might run .1 seconds slower on a Sparc machine! I think
anal programmers like me gravitate towards C++ because we like to show
how smart we are by naming off some arbitrary fact. Sure, it doesn't
make development any better, but it gives us something to waste memory
with. I find myself getting super excited when I get to use
