Re: Number of languages known [was Re: Python is readable] - somewhat OT
Re-trolling.
On Wed, Apr 4, 2012 at 1:49 AM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
As part of my troll-outreach effort, I will indulge here. I was
specifically thinking about some earlier claims that programming
languages as they currently exist are somehow inherently superior to a
formalized natural language in expressive power.
I would argue that they are, but only for the very limited purpose for
which they are written. With the possible exception of Inform 7, most
programming languages are useless at describing (say) human interactions.
I was thinking about this statement this morning. Compression is just
the process of finding a more efficient encoding for information. I
suppose you could say then that language developers could
theoretically be trying to compress natural language representations
of information. The problem then is that everyone is doing a horrible
job, because they are approaching the subject in an ad hoc manner.
There are multiple branches of mathematics and computer science that
deal with this exact subject in a rigorous way. The trick is to find
an encoding that has low space complexity, and for which the
transformation to knowledge is efficient, for human beings.
Lets assume that the input to be encoded are logical
(proposition/predicate) statements. The first thing that came to mind
when thinking this way is radix trees and directed acyclic word graphs
(a form of DFA). These structures are fairly easy to work out on
paper given a set of inputs, and it is fairly easy to reconstruct a
set of inputs from the structure. Perhaps, we could use natural
language statements, and some very minimal extended syntax to indicate
a data structure (which fans out to a set of statements). As a quick
example to see what I mean (mimicking some python syntax for
similarity):
in the context of chess:
a color is either white or black
the board:
is a cartesian grid having dimension (8, 8)
has squares, representing points on the grid
a square:
has a color
contains a piece or is empty
a piece:
has a color
is located in a square or has been captured
a { king, queen, rook, bishop, knight, pawn } is a type of piece
It should be clear that this is a directed acyclic phrase graph, and
if you select a phrase fragment, then one phrase fragment from each
child level until reaching a leaf, the concatenation of the phrase
fragments forms a logical phrase. Note that the set braces are
shorthand for multiple statements. This was really easy to write, and
I bet even non programmers would have little or no trouble
understanding what was going on. Additionally, I could make a full
statement elsewhere, and if we have an algorithm to transform to a
canonical phrase structure and merge synonyms, it could be inserted in
the phrase graph, just as neatly as if I had written it there in the
first place. The sexy thing about that, is that lets you take two
sets of propositional statements, and perform set theoretic operations
on them (union, complement, etc), and get a phrase graph structure out
at the end which looks just like a nice neat little program. You
could even get really crazy, if you could define equivalence relations
(other than the natural relation) for the union (Set1.A ~ Set2.B) as
that would let you compose the graphs in arbitrarily many ways. If
you're dealing processes, you would also want to be able to specify
temporal equivalence (Process1.T1 ~ Process2.T6).
--
http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Wed, Apr 4, 2012 at 1:49 AM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 03 Apr 2012 13:17:18 -0400, Nathan Rice wrote: I have never met a programmer that was not completely into computers. That leaves a lot unspecified though. You haven't looked hard enough. There are *thousands* of VB, Java, etc. code monkeys who got into programming for the money only and who have zero inclination to expand their skills or knowledge beyond that necessary to keep their job. Every programmer that I've ever met who got into it for the money has washed out within about five years. Sometimes they make a lateral move to project management, other times they end up as requirements analysts, and occasionally they become technical sales staff. The story is always the same - they do technical mediocre work, but get along well with their peers, so they are transitioned to a role that requires more people skills. I've never met someone who had both poor people skills and mediocre technical skills who actually kept their job. Go to programming blogs, and you will find many examples of some allegedly professional programmer selecting an arbitrary blog post to ask Pls sombody write me this code, where this code is either an utterly trivial question or a six month project. Honestly, I have seen that, but usually when I inspect closer it is an Indian ODesk or Rent-a-coder worker who oversold himself and is trying to cover his ass. As part of my troll-outreach effort, I will indulge here. I was specifically thinking about some earlier claims that programming languages as they currently exist are somehow inherently superior to a formalized natural language in expressive power. I would argue that they are, but only for the very limited purpose for which they are written. With the possible exception of Inform 7, most programming languages are useless at describing (say) human interactions. Human languages are optimised for many things, but careful, step-by-step algorithms are not one of them. This is why mathematicians use a specialist language for their problem domain, as do programmers. Human language is awfully imprecise and often ambiguous, it encourages implicit reasoning, and requires a lot of domain knowledge: You have to be careful when you bring mathematical notation into the picture. Remember that mathematics has developed over thousands of years, with developments shared in many languages. Greek letters serve the same purpose in math that latin and greek names serve in biology - they are neutral and avoid confusion with common names in living languages. Not everything about mathematical notation is good, and in some cases it suffers the same issues that programming does. Mathematicians have a tendency to be very terse, and although some greek letters and symbols have standard meaning, many authors run roughshod over them. Logic is somewhat better than math in this regard, logicians respect their notation and rarely deviate from the standard meaning of symbols. Things ARE getting better, but for the most part it is still kind of a mess. Also, I should clarify that I consider part of mathematical notation to be natural language, namely +/-/*, and rational numbers. People discover these things on their own, mathematics just provides rigor. It is considered bad form to use them in prose, but that is just an arbitrary style restriction; children intermix mathematical symbols and language all the time, as to older students taking notes in a variety of subjects. Joe snatched the hammer from Fred. Hey, he said, what are you doing? Don't you know that he'll hit the roof if he catches you with that? Are you trying to get me to write obfuscated code? You can write ambiguous garbage in any language. The crux of my view is that programming languages exist in part because computers in general are not smart enough to converse with humans on their own level, so we have to talk to them like autistic 5 year-olds. That was fine when we didn't have any other options, but all the pieces exist now to let computers talk to us very close to our own level, and represent information at the same way we do. I think you're dreaming. We (that is to say, human beings in general, not you and I specifically) cannot even talk to each other accurately, precisely and unambiguously all the time. Natural language simply isn't designed for that -- hence we have specialist languages like legal jargon, mathematics, and programming languages, for specialist purposes. Legalese is English with a ton of new words. Mathematics is older than most languages in current use and has a lot of baggage that is (very) slowly being dealt with. Programming really can't take the relaxed attitude about cleaning up notation and vocabulary that we see in math. Mathematicians spend a lot of time thinking, and the transcription of their thoughts is a relatively minor portion
Re: Number of languages known [was Re: Python is readable] - somewhat OT
Long personal note ahead. tl;dr version: Computers are such a large shift for human civilization that generally we dont get what that shift is about or towards. Another option: since *computers* are such a general device, there isn't just one notion. In the long run I expect computing science to transcend its parent disciplines, mathematics and logic, by effectively realizing a significant part of Leibniz's Dream of providing symbolic calculation as an alternative to human reasoning. (Please note the difference between mimicking and providing an alternative to: alternatives are allowed to be better.) A thinking machine. +1. Needless to say, this vision of what computing science is about is not universally applauded. On the contrary, it has met widespread --and sometimes even violent-- opposition from all sorts of directions. I mention as examples (0) the mathematical guild, which would rather continue to believe that the Dream of Leibniz is an unrealistic illusion Mathematics is not a closet guild, it is large and contentious. Ideas live and die in mathematics based on their fundamental truth. If there is some bold, sweeping statement it *MIGHT* be possible to prove or disprove, mathematicians will be all over it. just look at Fermat's last theorem and the Poincare conjecture if you want proof of this. (1) the business community, which, having been sold to the idea that computers would make life easier, is mentally unprepared to accept that they only solve the easier problems at the price of creating much harder one Most business people I know secretly love when they can sell a solution to one problem that creates new problems (and thus opportunities for new products!). The business term for this is an Upsell or Value-add. (2) the subculture of the compulsive programmer, whose ethics prescribe that one silly idea and a month of frantic coding should suffice to make him a life-long millionaire I love hacker culture, but it has been infected by the idea of entrepreneurship as a good in and of itself. Being a creator is a beautiful thing, go forth and make *art*. Improve the human condition. Make the world a better place. STFU about venture capital and stage 2 funding and minimum viable products; that sort of talk is a sure sign that you haven't created anything of actual value. (3) computer engineering, which would rather continue to act as if it is all only a matter of higher bit rates and more flops per second These guys are doing something that I find very uninteresting, but is absolutely necessary. Bravo I say. (4) the military, who are now totally absorbed in the business of using computers to mutate billion-dollar budgets into the illusion of automatic safety Nations will always try and be imperialist. At least drones and robot soldiers mean less human suffering. (5) all soft sciences for which computing now acts as some sort of interdisciplinary haven Digital humanities (outside of a VERY small set of projects) is a joke. Multimedia history presentations (and what not) are the domain of edutainment companies, not academia. (6) the educational business that feels that, if it has to teach formal mathematics to CS students, it may as well close its schools. I feel quite the opposite actually. At the really top notch computer science schools, there is a clear mathematical bent (though it is interdisciplinary). Places like MIT, Stanford, Berkeley, CMU, Cambridge, etc make a STRONG effort to separate the mathematical/theory of computation side and engineering side. At your average state college, the computer science department is just a hodgepodge, and you tend to see more graphics, applied computation and embedded/DSP type people. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
The building cabinets problem is interesting: 1. To actually build a cabinet, there's a lot of domain knowledge that's probably implicit in most circumstances. A carpenter might tell another carpenter which hinge to use, but they won't have to talk about why doors need hinges or how to do the assembly. 2. It's quite common for humans to use computer programs as part of the design process. 3. Often, the output of a CAD program (at the file level) is some sort of vector representation that only describes the end product (basic dimensions, etc.). I wonder if there are mini-languages out there that allow you to describe cabinets in a very descriptive way, where the description easily translates to the actual steps of building the cabinet, not just the final dimensions. I think if you were to describe the parts of the cabinet that needed to be assembled separately (and thus could be viewed as separate entities in some sense) and showed the cabinet as the composition of those parts, you would be on the right track. Being a mediocre carpenter, I can't really say anything conclusively here though :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Tue, Apr 3, 2012 at 1:40 AM, alex23 wuwe...@gmail.com wrote: On Apr 3, 2:55 pm, Nathan Rice nathan.alexander.r...@gmail.com wrote: I don't care what people do related to legacy systems. And that's what earns you the label 'architecture astronaut'. Legacy systems are _part_ of the problem; it's very easy to hold to a purist approach when you ignore the bulk of the domain that causes the issues. There's _never_ going to be an InfoTech3k where we just stop supporting older code. There are people who are paid pretty well to support crappy old COBOL apps, but I am not among them (nor are you, with very high likelihood), so your we is misplaced. For all intents and purposes that software exists in an alternate reality. Remember the tutorial on global vs local optimization I made previously? Let me distill it... If you are unwilling to endure pain to move towards a better world you will always be trapped in a sub-optimal situation. I do care about programmers that are too lazy to learn, and would be happy to ignore the fact that programming is hard for most people to learn, so they can continue not learning. Those programmers are scumbags. Wait, what? Programmers are both too lazy to learn and yet somehow happy that the skills they've acquired are too hard for most people to learn? So how did they learn them? And they're also somehow lazy because they have to learn multiple languages to be effective, rather than one mythical ur-language? In my 20 years as a software developer, I have _never_ encountered anyone trying to deliberately expand the knowledge gap. This isn't a priesthood. Did you miss the part where I said that most people who learn to program are fascinated by computers and highly motivated to do so? I've never met a BROgrammer, those people go into sales. It isn't because there aren't smart BROmosapiens (sadly, there are), they just couldn't give two shits about computers so programming seems like a colossal waste of time to them. It isn't about people scheming to dis-empower then plebs rather it is about people who don't want to move outside their comfort zone. You can talk about people learning multiple languages all you want, but for the most part they will be 10 descendants of ALGOL, with minor variations. Very few people are willing to tackle something like Haskell or ML if they weren't taught functional programming in university, though there are a few that view it as an endurance trial or mountain to climb. Those people get a pass on most of what I've said thus far. Just don't let me hear you complaining because some syntax is not C like enough for you. Whenever I hear that I want to strangle the self-serving 'tard that wrote it. When I see people defending C like syntax as optimal or somehow much more expressive, that makes me doubly irritated. These are the people who are selfishly defending the status quo because they're invested. Syntax is never the issue, it's the deeper semantics. Is the scoping of one C-like language the same as C? How does it differ? Why does it differ? Is the difference a fundamental implementation issue that you really need to know before you actually grok the language? Are functions first-class objects? Are they actual objects or some kind of magical stub? Can you extend those objects with properties? etc etc Syntax and semantics are both a big mess right now. That is why I always address them both. Every language tackles _so many_ things differently. It's not lazy to say that you prefer something to resemble/be based on a language you have experience with, that's human nature. If you're insistent that your non-typical syntax is so much better, the onus is on you to prove it, not to insist that the lack of uptake is 'laziness'. The winds of change generally blow for programming when generations of older programmers leave the workforce. Alan Kay was a smart man, viewing programming as an educational tool and designing for youth is absolutely the right way to do things. If you try to retrain older programmers, you are basically telling them they have to change decades of learning for a moderate (but not huge) productivity increase, so that programming is accessible to a much wider group of people. Much like with the terminal to GUI transition, you will have people attacking declarative natural language programming as a stupid practice for noobs, and the end of computing (even though it will allow people with much less experience to be more productive than them). And one again: code is _communication_. Not having to understand new optimal patterns for every single language is a Good Thing. Code is a horrible medium for communication. If it weren't, I wouldn't be trolling this thread. Don't try to delude people that our modern ALGOL derivatives are the best possible way to model knowledge (including process knowledge) to a computer, because that is a lie. Um, okay, I'll stop doing
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Tue, Apr 3, 2012 at 9:51 AM, rusi rustompm...@gmail.com wrote: On Apr 3, 5:39 pm, Nathan Rice nathan.alexander.r...@gmail.com wrote: Don't think underlying, instead think canonical. Ultimately, the answers to your questions exist in the world for you to see. How does a surgeon describe a surgical procedure? How does a chef describe a recipe? How does a carpenter describe the process of building cabinets? Aside from specific words, they all use natural language, and it works just fine. A carpenter describes his carpentry-process in English A CSist describes his programming-process in English (at least all my CS books are in English) A carpenter uses his tools -- screwdriver, saw, planer --to do carpentry A programmer uses his tools to to programming -- one of which is called 'programming language' Doing programming without programming languages is like using toenails to tighten screws I would argue that the computer is the tool, not the language. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Tue, Apr 3, 2012 at 11:01 AM, Ian Kelly ian.g.ke...@gmail.com wrote: On Tue, Apr 3, 2012 at 6:39 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: Did you miss the part where I said that most people who learn to program are fascinated by computers and highly motivated to do so? I've never met a BROgrammer, those people go into sales. It isn't because there aren't smart BROmosapiens (sadly, there are), they just couldn't give two shits about computers so programming seems like a colossal waste of time to them. I have never met the brogrammer stereotype. I have also never met the non-brogrammer stereotype of nerdy solitude (well, maybe once). That's all these things are -- stereotypes. Real programmers are much more complex. I have never met a programmer that was not completely into computers. That leaves a lot unspecified though. Computers require you to state the exact words you're searching for as well. Try looking again, and this time allow for sub-categories and synonyms, along with some variation in word order. Lazy troll. You made the claim. The onus is on you to provide the evidence. I reserve the right to be lazy :) As part of my troll-outreach effort, I will indulge here. I was specifically thinking about some earlier claims that programming languages as they currently exist are somehow inherently superior to a formalized natural language in expressive power. I think part of this comes from the misconception that terse is better (e.g. Paul Graham's thoughts on car/cdr), which doesn't take into account that your brain compresses frequently occurring English words VERY efficiently, so they actually take up less cognitive bandwidth than a much shorter non-word. This behavior extends to the phrase level as well; longer phrases that are meaningful in their own right take up less bandwidth than short nonsensical word combinations. On the semantic side, most people already understand branched processes and procedures with conditional actions pretty well. People program other people to perform tasks constantly, and have been doing so for the entirety of our existence. The problem occurs when programming language specific semantic artifacts must be considered. These artifacts are for the most part somewhat arbitrary, or you would see them frequently in other areas, and they wouldn't confuse people so much. I think the majority of these relate to how the computer operates internally - this is the stuff that really turns most people off to programming. The crux of my view is that programming languages exist in part because computers in general are not smart enough to converse with humans on their own level, so we have to talk to them like autistic 5 year-olds. That was fine when we didn't have any other options, but all the pieces exist now to let computers talk to us very close to our own level, and represent information at the same way we do. Projects like IBM's Watson, Siri, Wolfram Alpha and Cyc demonstrate pretty clearly to me that we are capable of taking the next step, and the resurgence of the technology sector along with the shortage of qualified developers indicates to me that we need to move now. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
A carpenter uses his tools -- screwdriver, saw, planer --to do carpentry A programmer uses his tools to to programming -- one of which is called 'programming language' Doing programming without programming languages is like using toenails to tighten screws I would argue that the computer is the tool, not the language. Computer science is as much about computers as astronomy is about telescopes -- E W Dijkstra Here are some other attempted corrections of the misnomer computer science: http://en.wikipedia.org/wiki/Computer_science#Name_of_the_field I view computer science as applied mathematics, when it deserves that moniker. When it doesn't, it is merely engineering. Ironically, telescopes are a tool that astronomers use to view the stars. On Tue, Apr 3, 2012 at 1:25 PM, rusi rustompm...@gmail.com wrote: All this futuristic grandiloquence: On Apr 3, 10:17 pm, Nathan Rice nathan.alexander.r...@gmail.com wrote: The crux of my view is that programming languages exist in part because computers in general are not smart enough to converse with humans on their own level, so we have to talk to them like autistic 5 year-olds. That was fine when we didn't have any other options, but all the pieces exist now to let computers talk to us very close to our own level, and represent information at the same way we do. Projects like IBM's Watson, Siri, Wolfram Alpha and Cyc demonstrate pretty clearly to me that we are capable of taking the next step, and the resurgence of the technology sector along with the shortage of qualified developers indicates to me that we need to move now. needs to be juxtaposed with this antiquated view I would argue that the computer is the tool, not the language. ... a view that could not be held by an educated person after the 1960s -- ie when it became amply clear to all that the essential and hard issues in CS are about software and not hardware I'll go ahead and forgive the club handed fallacies, so we can have a nice discussion of your primary point. What a civil troll I am :) Lets start with some analogies. In cooking, chefs use recipes to produce a meal; the recipe is not a tool. In architecture, a builder uses a blueprint to produce a building; the blueprint is not a tool. In manufacturing, expensive machines use plans to produce physical goods; the plans are not the tool. You could say the compiler is a tool, or a development environment is a tool. The programming language is a mechanism for communication. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Tue, Apr 3, 2012 at 4:20 PM, Terry Reedy tjre...@udel.edu wrote: On 4/3/2012 8:39 AM, Nathan Rice wrote: Ultimately, the answers to your questions exist in the world for you to see. How does a surgeon describe a surgical procedure? How does a chef describe a recipe? How does a carpenter describe the process of building cabinets? Aside from specific words, they all use natural language, and it works just fine. Not really. Surgeon's learn by *watching* a surgeon who knows the operation and next (hopefully) doing a particular surgery under supervision of such a surgeon, who watches and talks, and may even grab the instruments and re-show. They then really learn by doing the procedure on multiple people. They often kill a few on the way to mastery. Well, there is declarative knowledge and procedural knowledge. In all these cases, only the procedural knowledge is absolutely necessary, but the declarative knowledge is usually a prerequisite to learning the procedure in any sort of reasonable manner. I first learned basic carpentry and other skills by watching my father. I don't remember that he ever said anything about how to hold the tools. I similarly learned basic cooking by watching my mom. My knowledge of how to crack open an egg properly and separate the yolk from the rest is a wordless memory movie. A picture is worth a thousand words :) If you would like, I don't have any problem incorporating visual programming and programming by demonstration. I didn't go in that direction because I have enough to defend as it is. I like to look at it from the perspective of teaching/communicating, rather than operating a simple machine. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Sun, Apr 1, 2012 at 11:18 PM, alex23 wuwe...@gmail.com wrote: On Mar 30, 3:37 pm, Nathan Rice nathan.alexander.r...@gmail.com wrote: We live in a world where the tools that are used are based on tradition (read that as backwards compatibility if it makes you feel better) and as a mechanism for deriving personal identity. The world is backwards and retarded in many, many ways, this problem is interesting to me because it actually cuts across a much larger tract than is immediately obvious. Do you produce commercial code in a team? Because going by your absolutist bullshit here, it certainly doesn't sound like it. Think of me like the Wolf, the cleaner in pulp fiction that Marcellis Wallis calls in to take care of the mess when Jules accidentally blows a kid's brains out in the back of a car. I get called in when my skills are needed, and when the mess has been handled and things are back to normal I take my leave. When I join an organisation that requires language A as all of its systems are written in it, is that 'tradition' or 'personal identity'? How is 'compatibility' - either with existing systems or existing *developers* - a backwards and retarded approach to complex problems? I don't care what people do related to legacy systems. There will always be a COBOL. I do care about programmers that are too lazy to learn, and would be happy to ignore the fact that programming is hard for most people to learn, so they can continue not learning. Those programmers are scumbags. Just don't let me hear you complaining because some syntax is not C like enough for you. Whenever I hear that I want to strangle the self-serving 'tard that wrote it. When I see people defending C like syntax as optimal or somehow much more expressive, that makes me doubly irritated. These are the people who are selfishly defending the status quo because they're invested. If you're going to be selfish and inconsiderate at least be honest about it, rather than pretending that one of the earliest languages somehow got almost everything right and should be the basis for new languages till the end of time. This goes for most of the ALGOL derived languages. I don't have a problem if you know your language well and are happy using it, that's great. Don't try to delude people that our modern ALGOL derivatives are the best possible way to model knowledge (including process knowledge) to a computer, because that is a lie. If I've chosen language A because some aspect of its syntax maps better onto my mind (or for _whatever_ reason that makes individuals prefer one language to another), and you've chosen language B: who gets to decide which is the 'superior' language, which is the 'better' mapping etc? You should be able to live in your reality if you want, as long that doesn't impinge on others. Of course, if you disagree on basic grammar, then I would have to ask you, do you disagree about English grammar, or have you accepted it so that you can communicate with people? This is why I advocate following English grammar closely for syntax - people have accepted it and don't make a big deal, and it is the way we represent information already. You're arguing for a top-down centralised approach to language development that just will _never_ exist, simply because it cannot. If you don't accept that, I believe there's a fascinating fork called Python 4000 where your ideas would be readily adopted. You completely missed my point. In fact, my argument is for a bottom up approach, with a meeting point which is much closer than the machine code which is currently used. However you want to represent it, the knowledge is the same, and that is what matters. We need to get past the idea of different, incompatible languages, and settle on a common knowledge representation format that underlies all languages, and is compatible. If you want to make an alex23 DSL where up is down and inside is upside down, go for it, just as long as it is represented in a sensible set of semantic primes that I can transform to whatever reality I want. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On Sat, Mar 31, 2012 at 2:15 AM, Lie Ryan lie.1...@gmail.com wrote: On 03/21/2012 03:55 AM, Nathan Rice wrote: snip I think you've just described that greedy algorithm can't always find the globally optimal solution. Right. Using gradient descent on an algebraic surface is probably the most natural example of why this is the case, since balls rolling down a surface from a starting point to the bottom of a bowl is an exact analogy. On Sat, Mar 31, 2012 at 4:05 AM, Chris Angelico ros...@gmail.com wrote: On Sat, Mar 31, 2012 at 10:01 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: It seems to me that Indented blocks of text are used pretty frequently to denote definition bodies, section subordinate paragraphs and asides. The use of the colon seems pretty natural too. Parentheses are fairly natural for small asides. The notion of character delimiters for large sections of text is actually pretty unnatural with the exception of quotes. Perhaps in formal written English, but not in spoken, and often not in casual writing either. Play around with my actual example, an if clause, and see where the punctuation goes in English - and how easily you can construct ambiguous sentences. Sure an event has occurred recently if it occurred in the last time step. if xyz has occurred recently, that implies abc will occur in the next time step. when event abc occurs, all unbound floops become bound, and at most three newly bound floops are eaten by blargs. blargs that have not eaten in the last 3 time steps eat before blargs that have eaten in those time steps. Notice I don't talk about HOW anything is done, just the logic of what is happening. The computer should be capable of making an inventory of exactly what it will need to do given the statements I have made, and choose the best data structures and algorithms for the job. If we are in undecidable/halting problem territory (indefinite recursion) then the computer should at least be nice enough to tell you it is confused and would like some help. I don't like declarations, my personal preference is to have typed signatures, and implicit declaration with type inference elsewhere. I view it as a matter of personal preference though, the result should be the same, and it should be possible to view the code either way. I'm not sure what you mean by typed signatures, can you elaborate please? Just like the standard way in the Haskell community. To demonstrate using Python annotations... def myfunc(Sequence : a, Integral : b, Map : c) - Sequence: ... Given starting types and ending types, you can correctly infer some VERY complex internal types. Haskell will let you omit signature types as well, but that is really a bad idea because they add readability and you will have to add them often anyhow if you are dealing with complex types. Better to be consistent... As a funny aside, people usually provide input type and return type annotations to python functions, as part of the docstring (for sphinx). To be honest, I like having types baked into my code, though not nominal types (the sort that is an A because it was declared as an A or a subclass of A), but rather structural types (i.e. Abstract base classes, or Java interfaces, if you didn't have to add the implements ...). I don't like having to verbosely tell the computer about the types of everything I'm going to use, I only care that it gives me the output I want if I give it some agreed upon input. It should be smart enough to check the little things. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
Mathematics is all about abstraction. There are theories and structures in mathematics that have probably gone over a hundred years before being applied. As an analogy, just because a spear isn't useful while farming doesn't mean it won't save your life when you venture into the woods and come upon a bear. A spear is a concrete application of the principle of leverage, not an abstraction. I also point out leverage was discovered experimentally long before anyone had an abstraction for it. And an analogy is a device to demonstrate the fundamental character of an argument in a different context. In any case, so what? Who is saying that mathematics is useless? Not me, and not Joel Spolksy. You are hunting strawmen with your non-abstract spear. I don't think it is a strawman. He decries things that aren't immediately useful. That describes almost all pure math. If he had excluded things that have some characteristic of truth, and just talked about overly general systems, I might agree with him. Spolsky has written at least three times about Architecture Astronauts, and made it abundantly clear that the problem with them is that they don't solve problems, they invent overarching abstractions that don't do anything useful or important, and hype them everywhere. I believe in the idea of things should be as simple as possible, but not simpler. Programming as it currently exists is absolutely convoluted. I am called on to help people learn to program from time to time, and I can tell you that we still have a LONG way to go before programming approaches a global optimum in either the semantic or syntactic space. Never mind configuring a build or anything else related to projects. The whole setup really is horrible, and I'm convinced that most of the people who are capable of changing this are more concerned about their personal investment in the way things are than helping others. There are a few exceptions like Alan Kay, but mostly people want to graft shortcuts on to what already exists. You keep reading this as an assault on abstract mathematics, science and knowledge for its on sake. It isn't any of these things. I never said it was an attack on science. Scientists don't really do abstraction, they explain observations. Mathematicians are the ones who discover truth that may be completely disconnected from reality. If I'm paid to solve a problem, and instead I build an abstraction that doesn't help solve the problem, then I'm guilty of doing architecture astronauting. If I ignore everything Joel wrote and just use that definition, I agree with you. He is basically saying they are too clever for their own good, as a result of being fixated upon purely intellectual constructs. Yes, and he is right to do so, because that is the characteristic of the Architecture Astronaut: being so fixated on over-arching abstract concepts that, far from those abstractions making it easier to solve the problems they are being paid to solve, they actually make them harder. Good abstractions enable problems to be solved. Bad abstractions don't. If I ask you to build me a website, I probably don't want a website- builder, I certainly don't want a website-builder-generator, and I absolutely do not want you to generalise the concept of a compiler and create a whole new abstract language for describing meta-compilers so that I can create a brand new programming language for generating meta- compilers that build compilers that will build factories for building website generators so I can make my own website in just three easy steps (the simplest one of which is about twice as difficult as just building the website would have been). Again, I follow the principle of everything should be as simple as possible, but no simpler. I have in the past built website builders rather than build websites (and done a similar thing in other cases), but not because I am trying to discover some fundamental truth of website building, because that would be bullshit. I did it because building websites (or whatever else it was) is a boring, tedious problem, and a website builder, while being more work, is an engaging problem that requires thought. I enjoy challenging myself. If you are being paid to build abstractions in the ivory tower, on the chance that one in a thousand abstractions turns out to be a game changer, or just because of a love of pure knowledge, that's great. I love abstract mathematics too. I read maths in my free time, you won't find me saying that it is bad or useless or harmful. But does it solve real problems? You forget that even abstractions that never directly get turned into something real are almost invariably part of the intellectual discourse that leads to real things. Well, of course it does, and often in a most surprising places. But that's because out of the hundred thousand abstractions, we see the hundred that actually solve concrete problems. The
Re: Python is readable
On Fri, Mar 30, 2012 at 12:20 PM, Chris Angelico ros...@gmail.com wrote: On Sat, Mar 31, 2012 at 12:46 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: I believe in the idea of things should be as simple as possible, but not simpler. Programming as it currently exists is absolutely convoluted. I am called on to help people learn to program from time to time, and I can tell you that we still have a LONG way to go before programming approaches a global optimum in either the semantic or syntactic space. Aside from messes of installing and setting up language interpreters/compilers, which are averted by simply having several pre-installed (I think my Linux boxes come with some Python 2 version, Perl, bash, and a few others), that isn't really the case. Starting a Python script is easy. Programming gets convoluted only when, and to the extent that, the task does. It is true that program complexity is correlated with problem complexity, language and environment complexity is undeniable. If you want to prove this to yourself, find someone who is intelligent and has some basic level of computer literacy, sit them down at a computer and ask them to solve simple problems using programs. You could even be nice by opening the editor first. Don't help them, just watch them crash and burn. Then sit them in front of code that already works, and ask them to modify it to do something slightly different, and again just watch them crash and burn in all but the simplest of cases. It is painful - most of the time they just give up. These same people almost universally can describe the exact process of steps verbally or in writing to do what is required without any trouble; there might be some neglected edge cases, but if you describe the failed result, often times they will realize their mistake and be able to fix it quickly. Jeff Atwood had an article about programming sheep and non programming goats, and while he views it as a statement about people's failings, I view it as a statement about the failings of programming. Human computer interaction is really important, and the whole prefab GUI concept doesn't scale along any axis; people need to be able to interact with their computers in a flexible manner. In the beginning, we had no choice but to bend our communication to the the machine, but we're moving past that now. The machine should respect the communication of humans. We shouldn't decry natural language because of ambiguity; If we're ambiguous, the machine should let us know and ask for clarification. If we phrase things in a really awkward way, the machine should tell us so, and suggest a more understandable rendition (just like word processors do now). If the machine does something we didn't want based on our instructions, we should be able to state additional requirements in a declarative manner. Restricted natural languages are an active area of current research, and they clearly demonstrate that you can have an expressive formal language that is also valid English. Make no mistake about it, programming is a form of computer human interaction (I figured that would be an accepted mantra here). Think of it as modeling knowledge and systems instead of pushing bits around. You are describing things to the computer. To move from the syntactic domain to my point about programming languages, imagine if one person describes physics to a computer in French, and another person describes chemistry to a computer in English. The creators of the computer made different languages act as disjoint knowledge domains. The computer is incapable of making any inferences in physics which are informed by chemistry, and vice versa, unless someone comes along and re-describes one of the disciplines in the other language. Worse still, if someone that only speaks Mandarin comes along, the computer won't be able to tell him anything about either domain. Now imagine the creators of the computer decided that an acceptable solution was to have people print out statements from one domain in a given language, take it to another computer that scans the printout, translates it to a different language, and prints out the translated copy, then have that person take the translated copy back to the original computer, and scan it again in order to ask a cross cutting question. I hope from that perspective the paucity of our current methods will be more apparent. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
This is more a matter of being unable to express themselves appropriately. If I allowed them to write an exact process of steps to do what's required, those steps would either be grossly insufficient for the task, or would BE pseudo-code. There are plenty of people who cannot write those sorts of instructions at all. They're called non-programmers. Anyone who doesn't code but can express a task in such clear steps as you describe is what I would call a non-coding programmer - and such people are VERY easily elevated to full programmer status. I've worked with several like that, and the border between that kind of clear, concise, simple instruction list and actual Python or REXX code is so small as to be almost nonexistent. It's not the programming languages' fault. It's a particular jump in thinking that must be overcome before a person can use them. Your statement that the difference between Python or REXX and pseudo-code is almost non existent is completely false. While people reading Python might be able to guess with higher accuracy what a program does than some other programming languages, there is still a set of VERY specific set of glyphs, words and phrase structures it requires. Pretty much anyone can follow a recipe to make a meal (and there are a lot other examples of this), and conversely given the knowledge of how to make some dish, pretty much everyone could describe the process as a recipe. The same person will fail miserably when trying to create working code that is MUCH simpler from a conceptual standpoint. Non coders are not stupid, they just don't appreciate the multitude of random distinctions and computer specific abstractions programming foists on them for the purpose of writing EFFICIENT code. I'm talking about like multiple int/number types, umpteen different kinds of sequences, tons of different data structures that are used for basically the same things under different circumstances, indices starting at 0 (which makes amazing sense if you think like a machine, and very little if you think like a human), the difference between logical and bitwise operators (union and intersection would be better names for the latter), string encoding, etc. When you combine these with having to communicate in a new (very arbitrary, sometimes nonsensical) vocabulary that doesn't recognize synonyms, using an extremely restricted phrase structure and an environment with very limited interactivity, it should become clear that the people who learn to program are invariably fascinated by computers and very motivated to do so. I'm going to assume that you didn't mean that non coders are incapable of instructing others (even though your statement implies it directly). I think the ability of non coders to describe procedures would surprise you. Things I hear over and over from non coders all tie into people being frustrated that computers don't grasp similarity, and don't try to figure out what they want at all; most people provide instructions in an interactive manner. The computer is too stupid to interact with humans, so you have to tell it what to do, then try to run it, watch it fail, then go back and modify your program, which is highly unnatural. I think you'd find that these non coders would do very well if given the ability to provide instructions in a natural, interactive way. They are not failing us, we are failing them. Etcetera, etcetera. Everyone who's made the jump will see the benefit of the side they're on; most who haven't won't. Same with non-programmers to programmers. Why should I write like that when I could just write English? Simple: Because dedicated programming languages are far more expressive for the job. Really? Or could it be that algorithms for natural language processing that don't fail miserably is a very recent development, restricted natural languages more recent still, and pretty much all commonly used programming languages are all ~20+ years old? Could it also be that most programmers don't see a lot of incentives to make things accessible, since they're already invested in the status quo, and they might lose some personal value if programming stopped being an arcane practice? Creating a programming language is a time consuming and laborious process, the fact that people are doing it constantly is a clear indication that what we have is insufficient. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On Fri, Mar 30, 2012 at 4:20 PM, Chris Angelico ros...@gmail.com wrote: On Sat, Mar 31, 2012 at 6:55 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: I think you'd find that these non coders would do very well if given the ability to provide instructions in a natural, interactive way. They are not failing us, we are failing them. The nearest thing to natural-language command of a computer is voice navigation, which is another science that's plenty old and yet still current (I first met it back in 1996 and it wasn't new then). You tell the computer what you want it to do, and it does it. Theoretically. The vocabulary's a lot smaller than all of English, of course, but that's not a problem. The problem is that it's really REALLY slow to try to get anything done in English, compared to a dedicated domain-specific language (in the case of typical OS voice navigation, the nearest equivalent would probably be a shell script). I'm sure a ford truck would smoke a twin engine cessna if you compare their speed on the ground. Let the cessna fly and the ford doesn't have a snowball's chance. If you're navigating by going cee dee space slash somefolder slash some other folder slash some third folder slash semicolon emacs somename dash some other name dash something dot something else dot one the analogy would be a boss telling his secretary to reserve him a flight by saying visit site xyz, click on this heading, scroll halfway down, open this menu, select this destination, ... instead of book me a flight to San Jose on the afternoon of the 23rd, and don't spend more than $500. Totally. That's why we're all still programming in assembly language and doing our own memory management, because we would lose a lot of personal value if programming stopped being so difficult. If it weren't for all these silly new-fangled languages with their automatic garbage collection and higher order function handling, we would all be commanding much higher salaries. Did you miss the fact that a 50 year old programming language (which still closely resembles its original form) is basically tied for the title of currently most popular, and the 3 languages following it are both nominal and spiritual successors, with incremental improvements in features but sharing a large portion of the design. Programming language designers purposefully try to make their language C-like, because not being C-like disqualifies a language from consideration for a HUGE portion of programmers, who cower at the naked feeling they get imagining a world without curly braces. Fear of change and the unknown are brutal, and humans are cowardly creatures that will grasp at whatever excuses they can find not to acknowledge their weaknesses. I also mentioned previously, most developers are just trying to graft shortcut after shortcut on to what is comfortable and familiar because we're inherently lazy. Additionally, I'm quite certain that when we finally do have a method for programming/interacting with computers in a natural way, many people invested in previous methods will make snarky comments about how lame and stupid people using the new methods are, just like we saw with command line/keyboard elitists who make fun of people who prefer a mouse/gui, even though in most cases research showed that the people using the mouse/gui actually got work done faster. You can even look at some comments on this thread for evidence of this. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On Fri, Mar 30, 2012 at 5:45 PM, Chris Angelico ros...@gmail.com wrote: On Sat, Mar 31, 2012 at 7:58 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: Programming language designers purposefully try to make their language C-like, because not being C-like disqualifies a language from consideration for a HUGE portion of programmers, who cower at the naked feeling they get imagining a world without curly braces. Fear of change and the unknown are brutal, and humans are cowardly creatures that will grasp at whatever excuses they can find not to acknowledge their weaknesses. Braces are clear delimiters. English doesn't have them, and suffers for it. (Python's indentation is, too, but English doesn't have that either.) It's a lot harder to mark the end of an if block in English than in pretty much any programming language. It seems to me that Indented blocks of text are used pretty frequently to denote definition bodies, section subordinate paragraphs and asides. The use of the colon seems pretty natural too. Parentheses are fairly natural for small asides. The notion of character delimiters for large sections of text is actually pretty unnatural with the exception of quotes. And be careful of what has to be given up to gain your conveniences. I've used languages that demand variable declarations and ones that don't, and I'm very much a fan of the former. There are many benefits to being explicit about that. I don't like declarations, my personal preference is to have typed signatures, and implicit declaration with type inference elsewhere. I view it as a matter of personal preference though, the result should be the same, and it should be possible to view the code either way. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Wed, Mar 28, 2012 at 9:33 PM, Chris Angelico ros...@gmail.com wrote: On Thu, Mar 29, 2012 at 11:59 AM, Rodrick Brown rodrick.br...@gmail.com wrote: The best skill any developer can have is the ability to pickup languages very quickly and know what tools work well for which task. Definitely. Not just languages but all tools. The larger your toolkit and the better you know it, the more easily you'll be able to grasp the tool you need. The thing that bothers me is that people spend time and mental energy on a wide variety of syntax when the semantics are ~90% identical in most cases (up to organization). We would be better off if all the time that was spent on learning syntax, memorizing library organization and becoming proficient with new tools was spent learning the mathematics, logic and engineering sciences. Those solve problems, languages are just representations. Unfortunately, programming languages seem to have become a way to differentiate yourself and establish sub-cultural membership. All the cool kids are using XYZ, people who use LMN are dorks! Who cares about sharing or compatibility! Human nature is depressingly self-defeating. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Thu, Mar 29, 2012 at 10:03 AM, Chris Angelico ros...@gmail.com wrote: On Fri, Mar 30, 2012 at 12:44 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: We would be better off if all the time that was spent on learning syntax, memorizing library organization and becoming proficient with new tools was spent learning the mathematics, logic and engineering sciences. Those solve problems, languages are just representations. Different languages are good at different things. REXX is an efficient text parser and command executor. Pike allows live updates of running code. Python promotes rapid development and simplicity. PHP makes it easy to add small amounts of scripting to otherwise-static HTML pages. C gives you all the power of assembly language with all the readability of... assembly language. SQL describes a database request. Here's a thought experiment. Imagine that you have a project tree on your file system which includes files written in many different programming languages. Imagine that the files can be assumed to be contiguous for our purposes, so you could view all the files in the project as one long chunk of data. The directory and file names could be interpreted as statements in this data, analogous to in the context of somedirectory or in the context of somefile with sometype. Any project configuration files could be viewed as declarative statements about contexts, such as in xyz context, ignore those or in abc context, any that is actually a this. Imagine the compiler or interpreter is actually part of your program (which is reasonable since it doesn't do anything by itself). Imagine the build management tool is also part of your program in pretty much the same manner. Imagine that your program actually generates another program that will generate the program the machine runs. I hope you can follow me here, and further I hope you can see that this is a completely valid description of what is actually going on (from a different perspective). In the context of the above thought experiment, it should be clear that we currently have something that is a structural analog of a single programming metalanguage (or rather, one per computer architecture), with many domain specific languages constructed above that to simplify tasks in various contexts. The model I previously proposed is not fantasy, it exists, just not in a form usable by human beings. Are machine instructions the richest possible metalanguage? I really doubt it. Lets try another thought experiment... Imagine that instead of having machine instructions as the common metalanguage, we pushed the point of abstraction closer to something programmers can reasonably work with: abstract syntax trees. Imagine all programming languages share a common abstract syntax tree format, with nodes generated using a small set of human intelligible semantic primes. Then, a domain specific language is basically a context with a set of logical implications. By associating a branch of the tree to one (or the union of several) context, you provide a transformation path to machine instructions via logical implication. If implications of a union context for the nodes in the branch are not compatible, this manifests elegantly in the form of a logical contradiction. What does pushing the abstraction point that far up provide? For one, you can now reason across language boundaries. A compiler can tell me if my prolog code and my python code will behave properly together. Another benefit is that you make explicit the fact that your parser, interpreter, build tools, etc are actually part of your program, from the perspective that your program is actually another program that generates programs in machine instructions. By unifying your build chain, it makes deductive inference spanning steps and tools possible, and eliminates some needless repetition. This also greatly simplifies code reuse, since you only need to generate a syntax tree of the proper format and associate the correct context to it. It also simplifies learning languages, since people only need to understand the semantic primes in order to read anything. Of course, this describes Lisp to some degree, so I still need to provide some answers. What is wrong with Lisp? I would say that the base syntax being horrible is probably the biggest issue. Beyond that, transformations on lists of data are natural in Lisp, but graph transformations are not, making some things awkward. Additionally, because Lisp tries to nudge you towards programming in a functional style, it can be un-intuitive to learn. Programming is knowledge representation, and state is a natural concept that many people desire to model, so making it a second class citizen is a mistake. If I were to re-imagine Lisp for this purpose, I would embrace state and an explicit notion of temporal order. Rather than pretending it didn't exist, I would focus on logical and mathematical machinery necessary
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Thu, Mar 29, 2012 at 2:53 PM, Devin Jeanpierre jeanpierr...@gmail.com wrote: Agreed with your entire first chunk 100%. Woohoo! High five. :) Damn, then I'm not trolling hard enough ಠ_ಠ On Thu, Mar 29, 2012 at 1:48 PM, Nathan Rice nathan.alexander.r...@gmail.com wrote: transformations on lists of data are natural in Lisp, but graph transformations are not, making some things awkward. Eh, earlier you make some argument towards lisp being a universal metalanguage. If it can simulate prolog, it can certainly grow a graph manipulation form. You'd just need to code it up as a macro or function :p Well, a lisp-like language. I would also argue that if you are using macros to do anything, the thing you are trying to do should classify as not natural in lisp :) I'm really thinking here more in terms of a general graph reactive system here, matching patterns in an input graph and modifying the graph in response. There are a lot of systems that can be modeled as a graph that don't admit a nested list (tree) description. By having references to outside the nesting structure you've just admitted that you need a graph rather than a list, so why not be honest about it and work in that context from the get-go. Additionally, because Lisp tries to nudge you towards programming in a functional style, it can be un-intuitive to learn. I think you're thinking of Scheme here. Common Lisp isn't any more functional than Python, AFAIK (other than having syntactic heritage from the lambda calculus?) Common-Lisp does very much embrace state as you later describe, Scheme much less so (in that it makes mutating operations more obvious and more ugly. Many schemes even outlaw some entirely. And quoted lists default to immutable (rgh)). I find it interesting that John McCarthy invented both Lisp and the situation calculus. As for set/setq, sure, you can play with state, but it is verbose, and there is no inherent notion of temporal locality. Your program's execution order forms a nice lattice when run on hardware, that should be explicit in software. If I were to do something crazy like take the union of two processes that can potentially interact, with an equivalence relation between some time t1 in the first process and a time t2 in the second (so that you can derive a single partial order), the computer should be able to tell if I am going to shoot myself in the foot, and ideally suggest the correct course of action. Well, what sort of language differences make for English vs Mandarin? Relational algebraic-style programming is useful, but definitely a large language barrier to people that don't know any SQL. I think this is reasonable. (It would not matter even if you gave SQL python-like syntax, the mode of thinking is different, and for a good reason.) I don't think they have to be. You can view functions as names for temporally ordered sequence of declarative implication statements. Databases just leave out the logic (this is hyperbole, I know), so you have to do it in client code. I don't feel that a database necessarily has to be a separate entity, that is just an artifact of the localized, specialized view of computation. As stronger abstractions are developed and concurrent, distributed computation is rigorously systematized, I think we'll go full circle. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On Thu, Mar 29, 2012 at 9:44 AM, Albert van der Horst alb...@spenarnc.xs4all.nl wrote: In article mailman.896.1332440814.3037.python-l...@python.org, Nathan Rice nathan.alexander.r...@gmail.com wrote: http://www.joelonsoftware.com/articles/fog18.html I read that article a long time ago, it was bullshit then, it is bullshit now. The only thing he gets right is that the Shannon information of a uniquely specified program is proportional to the code that would be required to generate it. Never mind that if a Thank you for drawing my attention to that article. It attacks the humbug software architects. Are you one of them? I really liked that article. I read the first paragraph, remembered that I had read it previously and stopped. I accidentally remembered something from another Joel article as being part of that article (read it at http://www.joelonsoftware.com/items/2007/12/03.html). I don't really have anything to say on Joel's opinions about why people can or should code, their his and he is entitled to them. I feel they are overly reductionist (this isn't a black/white thing) and have a bit of luddite character to them. I will bet you everything I own the only reason Joel is alive today because of some mathematical abstraction he would be all too happy to discount as meaningless (because, to him, it is). Of course, I will give Joel one point: too many things related to programming are 100% hype, without any real substance; if his article had been about bullshit software hype and he hadn't fired the broadsides at the very notion of abstraction, I wouldn't have anything to say. Anyhow, if you ugh rock good caveman smash gazelle put in mouth make stomach pain go away meaning, here it is: Programs are knowledge. The reverse is not true, because programming is an infantile area of human creation, mere feet from the primordial tide pool from whence it spawned. We have a very good example of what a close to optimal outcome is: human beings - programs that write themselves, all knowledge forming programs, strong general artificial intelligence. When all knowledge is also programs, we will have successfully freed ourselves from necessary intellectual drudgery (the unnecessary kind will still exist). We will be able to tell computers what we want on our terms, and they will go and do it, checking in with us from time to time if they aren't sure what we really meant in the given context. If we have developed advanced robotics, we will simultaneously be freed from most manual labor. The only thing left for Joel to do will be to lounge about, being creative while eating mangos that were picked, packed, shipped and unloaded by robots, ordered by his computer assistant because it knows that he likes them, then delivered, prepared and served by more robots. The roadblocks in the path include the ability to deal with uncertainty, understand natural languages and the higher order characteristics of information. Baby steps to deal with these roadblocks are to explicitly forbid uncertainty, simplify the language used, and explicitly state higher order properties of information. The natural evolution of the process is to find ways to deal with ambiguity, correctly parse more complex language and automatically deduce higher order characteristics of information. Clearly, human intelligence demonstrates that this is not an impossible pipe dream. You may not be interested in working towards making this a reality, but I can pretty much guarantee on the scale of human achievement, it is near the top. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
On Thu, Mar 29, 2012 at 7:37 PM, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Thu, Mar 29, 2012 at 3:50 PM, Nathan Rice nathan.alexander.r...@gmail.com wrote: Well, a lisp-like language. I would also argue that if you are using macros to do anything, the thing you are trying to do should classify as not natural in lisp :) You would run into disagreement. Some people feel that the lisp philosophy is precisely that of extending the language to do anything you want, in the most natural way. That is some people's lisp philosophy, though I wouldn't say that is a universal. Just like I might say my take on python's philosophy is keep it simple, stupid but others could disagree. At least, I disagree, but my lisp thoughts are the result of indoctrination of the Racket crowd. I don't know how well they represent the larger lisp community. But you should definitely take what I say from the viewpoint of the sort of person that believes that the whole purpose of lisps is to embed new syntax and new DSLs via macros. Without macros, there's no point of having this despicable syntax (barring maybe pedagogy and other minor issues). Heh, I think you can have a homoiconic language without nasty syntax, but I won't get into that right now. I'm really thinking here more in terms of a general graph reactive system here, matching patterns in an input graph and modifying the graph in response. There are a lot of systems that can be modeled as a graph that don't admit a nested list (tree) description. By having references to outside the nesting structure you've just admitted that you need a graph rather than a list, so why not be honest about it and work in that context from the get-go. I don't see any issue in defining a library for working with graphs. If it's useful enough, it could be added to the standard library. There's nothing all that weird about it. Graphs are the more general and expressive data structure, I think if anything you should special case the less general form. Also, most representations of graphs are precisely via a tree-like non-recursive structure. For example, as a matrix, or adjacency list, etc. We think of them as deep structures, but implement them as flat, shallow structures. Specialized syntax (e.g. from macros) can definitely bridge the gap and let you manipulate them in the obvious way, while admitting the usual implementation. We do a lot of things because they are efficient. That is why gaussian distributions are everywhere in statistics, people approximate nonlinear functions with sums of kernels, etc. It shouldn't be the end goal though, unless it really is the most expressive way of dealing with things. My personal opinion is that graphs are more expressive, and I think it would be a good idea to move towards modeling knowledge and systems with graphical structures. I don't think they have to be. You can view functions as names for temporally ordered sequence of declarative implication statements. Databases just leave out the logic (this is hyperbole, I know), so you have to do it in client code. I don't feel that a database necessarily has to be a separate entity, that is just an artifact of the localized, specialized view of computation. As stronger abstractions are developed and concurrent, distributed computation is rigorously systematized, I think we'll go full circle. Maybe I'm too tired, but this went straight over my head, sorry. Perhaps you could be a bit more explicit about what you mean by the implications/logic? Well, the curry howard correspondance says that every function can be seen as a named implication of outputs given inputs, with the code for that function being a representation of its proof. Since pretty much every function is a composition of many smaller functions, this holds down to the lowest level. Even imperative statements can be viewed as functions in this light, if you assume discrete time, and view every function or statement as taking the state of the world at T as an implicit input and returning as an implicit output the state of the world at T+1. Thus, every function (and indeed pretty much all code) can be viewed as a named collection of implication statements in a particular context :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
He did no such thing. I challenge you to find me one place where Joel has *ever* claimed that the very notion of abstraction is meaningless or without use. When great thinkers think about problems, they start to see patterns. They look at the problem of people sending each other word-processor files, and then they look at the problem of people sending each other spreadsheets, and they realize that there's a general pattern: sending files. That's one level of abstraction already. Then they go up one more level: people send files, but web browsers also send requests for web pages. And when you think about it, calling a method on an object is like sending a message to an object! It's the same thing again! Those are all sending operations, so our clever thinker invents a new, higher, broader abstraction called messaging, but now it's getting really vague and nobody really knows what they're talking about any more. Blah. When you go too far up, abstraction-wise, you run out of oxygen. Sometimes smart thinkers just don't know when to stop, and they create these absurd, all-encompassing, high-level pictures of the universe that are all good and fine, but don't actually mean anything at all. To me, this directly indicates he views higher order abstractions skeptically, and assumes because he does not see meaning in them, they don't hold any meaning. Despite Joel's beliefs, new advances in science are in many ways the result of advances in mathematics brought on by very deep abstraction. Just as an example, Von Neumann's treatment of quantum mechanics with linear operators in Hilbert spaces utilizes very abstract mathematics, and without it we wouldn't have modern electronics. I'm 100% behind ranting on software hype. Myopically bashing the type of thinking that resulted in the computer the basher is writing on, not so much. If he had said if you're getting very high up, find very smart people and talk to them to make sure you're not in wing nut territory I could have given him a pass. I really wish people wouldn't try to put Joel up on a pedestal. The majority of his writings either seem like sensationalist spins on tautological statements, self aggrandizement or luddite trolling. At least Stephen Wolfram has cool shit to back up his ego, Fog Creek makes decent but overpriced debuggers/version control/issue trackers... From my perspective, Stack Overflow is the first really interesting thing Joel had his hand in, and I suspect Jeff Atwood was probably the reason for it, since SO doesn't look like anything Fog Creek ever produced prior to that. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
He did no such thing. I challenge you to find me one place where Joel has *ever* claimed that the very notion of abstraction is meaningless or without use. [snip quote] To me, this directly indicates he views higher order abstractions skeptically, Yes he does, and so we all should, but that's not the claim you made. You stated that he fired the broadsides at the very notion of abstraction. He did no such thing. He fired a broadside at (1) software hype based on (2) hyper-abstractions which either don't solve any problems that people care about, or don't solve them any better than more concrete solutions. Mathematics is all about abstraction. There are theories and structures in mathematics that have probably gone over a hundred years before being applied. As an analogy, just because a spear isn't useful while farming doesn't mean it won't save your life when you venture into the woods and come upon a bear. and assumes because he does not see meaning in them, they don't hold any meaning. You are making assumptions about his mindset that not only aren't justified by his comments, but are *contradicted* by his comments. He repeatedly describes the people coming up with these hyper-abstractions as great thinkers, clever thinkers, etc. who are seeing patterns in what people do. He's not saying that they're dummies. He's saying that they're seeing patterns that don't mean anything, not that the patterns aren't there. He is basically saying they are too clever for their own good, as a result of being fixated upon purely intellectual constructs. If math was a failed discipline I might be willing to entertain that notion, but quite the opposite, it is certainly the most successful area of study. Despite Joel's beliefs, new advances in science are in many ways the result of advances in mathematics brought on by very deep abstraction. Just as an example, Von Neumann's treatment of quantum mechanics with linear operators in Hilbert spaces utilizes very abstract mathematics, and without it we wouldn't have modern electronics. I doubt that very much. The first patent for the transistor was made in 1925, a year before von Neumann even *started* working on quantum mechanics. The electronic properties of silicon (among other compounds) is an obvious example of where quantum theory provides for us. We might have basic circuits, but we wouldn't have semiconductors. In general, theory *follows* practice, not the other way around: parts of quantum mechanics theory followed discoveries made using the transistor: You do need data points to identify an explanatory mathematical structure. The Romans had perfectly functioning concrete without any abstract understanding of chemistry. If we didn't have QM, we'd still have advanced electronics. Perhaps not *exactly* the electronics we have now, but we'd have something. We just wouldn't understand *why* it works, and so be less capable of *predicting* useful approaches and more dependent on trial-and-error. Medicine and pharmaceuticals continue to be discovered even when we can't predict the properties of molecules. The stochastic method, while useful, is many orders of magnitude less efficient than analytically closed solutions. Not having access to closed form solutions would have put us back hundreds of years at least. My aunt makes the best damn lasagna you've ever tasted without any overarching abstract theory of human taste. And if you think that quantum mechanics is more difficult than understanding human perceptions of taste, you are badly mistaken. Taste is subjective, and your aunt probably started from a good recipe and tweaked it for local palates. That recipe could easily be over a hundred years old. An overarching mathematical theory of human taste/mouth perception, if such a silly thing were to exist, would be able to generate new recipes that were perfect for a given person's tastes very quickly. Additionally, just to troll this point some more (fun times!), I would argue that there is an implicit theory of human taste (chefs refer to it indirectly as gastronomy) that is very poorly organized and lacks any sort of scientific rigor. Nonetheless, enough empirical observations about pairings of flavors, aromas and textures have been made to guide the creation of new recipes. Gastronomy doesn't need to be organized or rigorous because fundamentally it isn't very important. In any case, Spolsky is not making a general attack on abstract science. Your hyperbole is completely unjustified. The mathematics of the 20th century, (from the early 30s onward) tend to get VERY abstract, in just the way Joel decries. Category theory, model theory, modern algebraic geometry, topos theory, algebraic graph theory, abstract algebras and topological complexes are all very difficult to understand because they seem so incredibly abstract, yet most of them already have important applications. I'm 100% positive if you
Re: Number of languages known [was Re: Python is readable] - somewhat OT
Here's a thought experiment. Imagine that you have a project tree on your file system which includes files written in many different programming languages. Imagine that the files can be assumed to be contiguous for our purposes, so you could view all the files in the project as one long chunk of data. The directory and file names could be interpreted as statements in this data, analogous to in the context of somedirectory or in the context of somefile with sometype. Any project configuration files could be viewed as declarative statements about contexts, such as in xyz context, ignore those or in abc context, any that is actually a this. Imagine the compiler or interpreter is actually part of your program (which is reasonable since it doesn't do anything by itself). Imagine the build management tool is also part of your program in pretty much the same manner. Imagine that your program actually generates another program that will generate the program the machine runs. I hope you can follow me here, and further I hope you can see that this is a completely valid description of what is actually going on (from a different perspective). [...] What does pushing the abstraction point that far up provide? I see why you are so hostile towards Joel Spolsky's criticism of Architecture Astronauts: you are one of them. Sorry Nathan, I don't know how you breathe that high up. For what it's worth, your image of everything from the compiler on up is part of your program describes both Forth and Hypercard to some degree, both of which I have used and like very much. I still think you're sucking vacuum :( We live in a world where the tools that are used are based on tradition (read that as backwards compatibility if it makes you feel better) and as a mechanism for deriving personal identity. The world is backwards and retarded in many, many ways, this problem is interesting to me because it actually cuts across a much larger tract than is immediately obvious. People throughout history have had the mistaken impression that the world as it existed for them was the pinnacle of human development. Clearly all of those people were tragically deluded, and I suspect that is the case here as well. -- http://mail.python.org/mailman/listinfo/python-list
Re: Number of languages known [was Re: Python is readable] - somewhat OT
Logo. It's turtles all the way down. -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
I will use = to mean is equivalent to. That's not part of the DSL.
A flow has one or more streams:
1 stream:
[1,2,3]
2 streams:
[1,3,5] | [2,4,6]
Two flows can be concatenated:
[1,2,3] + [4,5,6] = [1,2,3,4,5,6]
[0] + ([1,2] | [3,4]) + [10] = [0,1,2,10] | [0,3,4,10]
([1,2] | [10,20]) + ([3,4] | [30,40]) = [1,2,3,4] | [10,20,30,40]
Algebraically, your concatenation rules don't really make sense - your
flows are both distributive and non distributive. You also make the
implicit assumption of an order over streams in a flow, but disregard
the implications of that assumption in some cases. I understand what
you're trying to communicate, so I think you need to be a little more
strict and explicit in your definitions.
A flow can be transformed:
[1,2] - f = [f(1),f(2)]
([1,2] | [3,4]) - f = [f(1,3),f(2,4)]
([1,2] | [3,4]) - [f] = [f(1),f(2)] | [f(3),f(4)]
([1,2] | [3,4]) - [f,g] = [f(1),f(2)] | [g(3),g(4)]
[1,2] - [f,g] = [f(1),f(2)] | [g(1),g(2)]
Given the examples you pose here, it is clear that you are assuming
that the streams are synchronized in discrete time. Since you do not
provide any mechanism for temporal alignment of streams you are also
assuming every stream will have an element at every time point, the
streams start at the same time and are of the same length. Is this
what you want? These seem like pretty rough simplifying assumptions.
Some functions are special and almost any function can be made special:
[1,2,3,4,5] - filter(isprime) = [2,3,5]
[[],(1,2),[3,4,5]] - flatten = [1,2,3,4,5]
Note that 'filter' is not really necessary, thanks to 'flatten'.
This implies that your transformations again produce flows. You
should explicitly state this.
Flows can be named, remembered and used
as a value:
[1,2,3,4,5] - 'flow' + val('flow') = [1,2,3,4,5]*2
Is this a flow with two identical streams, or a flow with one long
stream formed by concatenation?
as a transformation chain:
[1,2,3] - skipfirst - 'again' | [4,5,6] - func('again')
= [2,3] | [5,6]
Recursion is also possible and stops when a function is applied to an empty
sequence.
Flows can be saved (push) and restored (pop) :
[1,2,3,4] - push - by(2) - 'double' - pop | val('double')
= [1,2,3,4] | [2,4,6,8]
There are easier ways to achieve the same result, of course:
[1,2,3,4] - [id, by(2)]
You are grasping at an algebra here, a sort of calculus of temporal
observations. You need to step back and make it rigorous before you
worry about issues such as a readable syntax.
Nathan
--
http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
I will use = to mean is equivalent to. That's not part of the DSL. A flow has one or more streams: 1 stream: [1,2,3] 2 streams: [1,3,5] | [2,4,6] Two flows can be concatenated: [1,2,3] + [4,5,6]= [1,2,3,4,5,6] [0] + ([1,2] | [3,4]) + [10]= [0,1,2,10] | [0,3,4,10] ([1,2] | [10,20]) + ([3,4] | [30,40])= [1,2,3,4] | [10,20,30,40] Algebraically, your concatenation rules don't really make sense - your flows are both distributive and non distributive. You also make the implicit assumption of an order over streams in a flow, but disregard the implications of that assumption in some cases. I understand what you're trying to communicate, so I think you need to be a little more strict and explicit in your definitions. When concatenating, either there are the same number of streams, or one of them is a single stream which is duplicated. Therefore, in this example: [0] + ([1, 2] | [3, 4]) you're concatenating a single stream with a pair, so the single stream is duplicated: ([0] | [0]) + ([1, 2] | [3, 4]) and then they can be concatenated: ([0, 1, 2] | [0, 3, 4]) However, this: ([0, 1] | [2, 3]) + ([4, 5] | [6, 7] | [8, 9]) won't work. I understand how he wants the system to work in this case; my point was that it isn't consistent. He stated flows can be concatenated, so [0] is just a flow of a single stream. Because he clearly intends that streams in a flow are ordered, that indicates that he wants some sort of algebraic module. He could get close if he can extend a stream to meet the requirements of a ring, then a flow would be a module over the ring of streams. The problem he is going to run into is that operations on streams as he defines them are not commutative. Identity elements for the two binary operations are also not immediately obvious to me. He could just be smart and use the pi calculus, it is rigorously developed and can model his desired behavior, if he reformulates it slightly. Another option is just to give up on being rigorous. Given the abstract treatment he attempted I would be disappointed, but if his only goal is to get practice writing code and he isn't interested in exploring the conceptual space of the problem domain it would be the right decision. -- http://mail.python.org/mailman/listinfo/python-list
Re: Stream programming
I understand what you're trying to communicate, so I think you need to be a little more strict and explicit in your definitions. No, I don't think you understand what I meant. I don't agree. Sorry. Yes. I thought that streams as an alternative to functional programming were widely known. Streams aren't really a paradigm of computation. They're a semantic element of a computational system which cuts across paradigms. If you want to retract that and say you were talking about dataflow programming (which is much larger than streams, and actually has a cohesive definition), I won't hold it against you. Ultimately though, functional programming and dataflow programming are closely linked, the main difference being the use of queues based rather than stacks. Isn't that obvious? BTW, those are not rigorous definitions. I thought I was talking to people who regularly works with streams. That is the GPU mailing list, down the hall on the left. Instead of talking of what I wasn't trying to do and, indeed, I didn't do, you should try to understand what I wanted to do and, in fact, I did. I'm afraid your cup is too full to understand simple things as the one I wrote in my OP. Clearly, because I didn't explicitly include the possibility that you are just writing throwaway code with no attempt at development of ideas for the purpose of practicing writing code in the paragraph after the one you quoted. If your goal is to learn to code, instead of posting a message stating that you have a superior way to compose code, you might want to try to solve a problem and ask others their opinion of the structure and techniques in your code. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
If I'm reading you correctly, you're expressing frustration with the state of language syntax unification in 2012. You mention language in a broad sense (not just programming languages, but also English, math, logic, etc.), but even in the narrow context of programming languages, the current state of the world is pretty chaotic. And this is a good thing. Programming languages are chaotic because the universe of programming problems is chaotic, and the strategies available to solve those problems are many and varied. Different programming languages are good for different things because they have been designed to work in different problem/solution spaces. Although I dislike C with a passion, I do recognise that it is good for when the programmer needs fine control over the smallest details. It is, after all, a high-level assembler. Likewise for Forth, which lets you modify the compiler and language as you go. There is a concept in statistical/mathematical modeling called minimum message length (a close analog is minimum description length), which asserts that the optimum model for some set of information is the one that minimizes the sum of the length of the model and the length of the set described by that model. Clearly no model is going to be optimal for every set of information. What I was alluding to in the post that Steve Howell replied to was that we need to have a programming language that is a model of models, then include a second order model as part of the program. Having one core language with many DSLs that can interoperate is infinitely better than having many languages that cannot. A language designed in such a way would also prevent issues like the Python 2 - 3 fiasco, because two versions of a DSL can be separate, and code can reference them independently while being able to interoperate. Some languages are optimized for the compiler, some for the writer, and some for the reader. So are optimized for numeric work, others for database access. Some are Jack-Of-All-Trades. Each language encourages its own idioms and ways of thinking about programming. The difference between compiler optimized and writer optimized languages is how many assertions they require about the thing being modeled to be a valid program, given its deductive rules. Ideally, the number of assertions should be flexible, and the interpreter/compiler should try and figure out the best way to implement it given the information it has. When it comes to programming, I say, let a thousand voices shout out. Instead of imagining a single language so wonderful that every other language is overshadowed and forgotten, imagine that the single language is the next Java, or C, or even for that matter Python, but whatever it is, it's not ideal for the problems you care about, or the way you think about them. Not so attractive now, is it? I agree about letting a thousand voices shout out, in the context of an embedding language that guarantees code interoperability. Additionally, the size of optimal DSLs for different areas is actually quite small, yet having separate languages for each DSL requires the user to re-learn common patterns such as collection manipulation, IO, etc. Pretty horrible all around. The optimistic view is that there will be some kind of inflection point around 2020 or so. I could imagine a perfect storm of good things happening, like convergence on a single browser platform, You call that a perfect storm of good things. I call that sort of intellectual and software monoculture a nightmare. The cores of English and math are pretty much singularly represented. At more nuanced or abstract levels, there is divergence in order to simplify the process of describing complicated things. How would you like it if linear algebra or algebraic geometry re-invented addition, multiplication, etc with completely different syntax and semantics (I'm ignoring non-commutativity of vector space multiplication) I want a dozen browsers, not one of which is so common that web designers can design for it and ignore the rest, not one browser so common that nobody dares try anything new. How about one browser that is infinitely configurable? That way if someone tries something new, you can try it as well without precluding anything else you already do. The CLI/JVM provide some flexibility, but they are not the correct path. They model the executable machine representations of language, rather than modeling the meta structure of language. This means that you still lack flexibility and dynamism. At least things are moving in the right direction. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
On Thu, Mar 22, 2012 at 9:17 AM, Chris Angelico ros...@gmail.com wrote: On Thu, Mar 22, 2012 at 11:47 PM, Nathan Rice nathan.alexander.r...@gmail.com wrote: Having one core language with many DSLs that can interoperate is infinitely better than having many languages that cannot. A language designed in such a way would also prevent issues like the Python 2 - 3 fiasco, because two versions of a DSL can be separate, and code can reference them independently while being able to interoperate. This is either utterly impossible, or already available, depending on your point of view. Of course it is already available. It is called the laws of physics. Everything we know of inter-operates in the context of physical reality perfectly. To have an infinitely-configurable program, you must make its configuration equivalent to writing code. There is already an infinitely-configurable program in Unix: gcc config.c; ./a.out takes a simple-format text file called config.c and processes it. It is true that infinite configuration requires that the configuration be specified in the language of the program. What's the problem? You want infinite DSLs? Behold! $ ./module1 | ./module2 | ./module3 | ./module4 Each one has a shebang that tells you what they are (Python 2, Python 3, something else entirely). There's a very strict interoperability protocol between the modules - they pass information around through stdout and stdin. You can write any module in any language, and you can rewrite module1 for Python 3 without affecting any other or the invocation sequence. It has always been possible to get from one point in space to another point in space in finite time. Was the invention of the automobile was redundant and unimportant? Obviously not, the characteristics of the process matter. For example, your ability to reason about the behavior of the system you posited as a whole is limited. Are there parts of the different modules that can execute concurrently? Is the output of module1 guaranteed to be acceptable as the input for module2? Is part of module3 redundant (and thus avoidable) given some conditions in module1? If you make a change in module2 what effect does that have on module3 and module4? What if you need to go back and forth between module2 and module3 until some criterion is met before you transition to module4? What if you sometimes want to run module2b instead of module2? Problems always come when you want to share data between dissimilar modules. There's no single perfect data-transfer protocol that works across all languages. What does it mean to call a function? If you truly want this level of flexibility, the best thing to do is to separate sections cleanly. In fact, the pipe system I show above could be used that way, although it's stupidly ugly. Everything going on stdout/stdin has a two-word dispatch envelope with a from and a to, and each module reads a line, and if it's not addressed to it, passes it on to stdout. Responses get sent out with their from and to reversed. All you need is for the last module's stdout to be linked back into the first module's stdin (not sure if you can do that or not - it has the feeling of plugging a power strip into itself), and it'll work perfectly. Add a bit of boilerplate so that the application developers don't see what's happening underneath, and you could pretend that dissimilar languages are able to call into each other. Doesn't mean it's efficient though! What is a function? You could say that it is a sequence of logical statements that relates some assertion with another. You can make a big deal about call by name versus call by value, but it only really matters on a conceptual level when dealing with infinite objects that do not have an analytic closure. You can make a big deal about data format, but scientists have been communicating using different unit systems for hundreds of years, I don't see assertions of relationship between structures as different from unit conversions from imperial to metric; it's all data. There are of course a few snags such as cardinality of the set represented by the integer data type in one case versus another, but that is a low level detail that would be a problem if DSLs were embedded in an expressive modelling language. Data structures ultimately boil down to assertions, assertions about those assertions and relations between assertions. TLDR version: Infinite flexibility means you're doing nothing. I could address that from a number of ways. First off, anything that is Turing complete is basically infinitely flexible but the difference in expressiveness (as measured by the combined length of the program and its input required to generate some output) varies over orders of magnitudes. So, we're actually discussing infinitely expressive though in fact we don't need to appeal to infinity here, so that is a strawman. The reason for that is the system in which
Re: Python is readable
There is a concept in statistical/mathematical modeling called minimum message length (a close analog is minimum description length), which asserts that the optimum model for some set of information is the one that minimizes the sum of the length of the model and the length of the set described by that model. (1) Optimum in what sense? Oh, I don't know, the information theoretic one? Of course, if you're part of the department of redundancy department, that might seem to appear to look like a poor, weak, sub-optimal criterion for the purposes of evaluation of optimality, goodness and positive quality. If you're getting hung up on the fact that there are deep ties to data compression in this language, let me help you out. Imagine that instead of having a simple Boolean algebra of two values, the possible values ranged over a set of elementary concepts. You are then trying to come up with a set of compound concepts and a permutation of those concepts that can describe (2) What does this have to do with designing programming languages? A program is a representation of knowledge (i.e. a model) which a machine has the ability to interpret. Let me know if you can't understand this, I'll break it down further for you. This discussion sounds to me like the apocryphal story about the spherical cow. http://en.wikipedia.org/wiki/Spherical_cow When you take the baton, you're supposed to keep running along the path in the same direction as the person who handed it to you. Clearly no model is going to be optimal for every set of information. What I was alluding to in the post that Steve Howell replied to was that we need to have a programming language that is a model of models, then include a second order model as part of the program. Having one core language with many DSLs that can interoperate is infinitely better than having many languages that cannot. Some people, when confronted with a problem, think I know, I'll use a DSL. Now they have two problems. Sure. When confronted with the problem of finding the area under a curve, using integral calculus is going to make things worse... I should just manually sum the infantesimals, from now until the end of time. And some people think I know, I'll use a meta-language with an infinite number of infinitely variable DSLs. Now they have an infinite number of problems. I'll agree with you, if we change that statement to say an infinite number of possible problems, all isomorphic to each other. A language designed in such a way would also prevent issues like the Python 2 - 3 fiasco, What fiasco? The extremely slow uptake? I don't really care one way or another but a lot of the devs have lamented the issue. You've just lost an awful lot of credibility with me. I didn't lose anything, technically you lost some of your belief in my credibility. So, tautologically, you are the one that lost in this situation. I'm just surprised you didn't manage to fit quantum mechanics and the interconnectedness of all things into it :) Math/Logic are encompassing. I could draw analogies to quantum theory if you really wanted (category theory is great for this sort of thing). As for the interconnectedness of all things, that is quack language, I do science. TL;DR there are a huge number of incompatible programming languages because people are modeling a permutation rather than the input to a permutation generating function. No offence Nathan, but I think you need to come back down to earth for air before you black out: http://www.joelonsoftware.com/articles/fog18.html I read that article a long time ago, it was bullshit then, it is bullshit now. The only thing he gets right is that the Shannon information of a uniquely specified program is proportional to the code that would be required to generate it. Never mind that if a program meets a specification, you shouldn't care about any of the values used for unspecified parts of the program. If you care about the values, they should be specified. So, if Joel had said that the program was uniquely specified, or that none of the things that weren't specified require values in the programming language, he might have been kinda, sorta right. Of course, nobody cares enough to specify every last bit of minutiae in a program, and specifications change, so it is pretty much impossible to imagine either case ever actually occurring. Or at least before *I* black out. Even if somebody manages to write your meta-language, you're going to run into the problem of who is going to be able to use it. The typical developer knows three, maybe four languages moderately well, if you include SQL and regexes as languages, and might have a nodding acquaintance with one or two more. Please don't black out Steven. I'm trying to encourage thought in interesting directions; you can't think when you're unconscious. Lets draw the analogy to mathematicians. If a mathematician wants to
Re: Python is readable
For example, your ability to reason about the behavior of the system you posited as a whole is limited. Are there parts of the different modules that can execute concurrently? Is the output of module1 guaranteed to be acceptable as the input for module2? Is part of module3 redundant (and thus avoidable) given some conditions in module1? If you make a change in module2 what effect does that have on module3 and module4? What if you need to go back and forth between module2 and module3 until some criterion is met before you transition to module4? What if you sometimes want to run module2b instead of module2? Pipes allow different modules to execute concurrently (except on DOS and maybe Windows, haven't checked). Nothing in ANY language guarantees acceptable input, but protocolling would do that for you. If part of module3 is redundant, you don't call it. If you make a change to one that affects others, then you fix the others, same as in any other coding system (or non-coding system - if you upgrade your hardware from an 8086 with 1MB of memory to a modern box with 4GB, you'll have to rewrite your code to take advantage of it). The (somewhat stupid) protocol I suggested allows you to go between 2 and 3 before going to 4. If you want module2b, you add it to the chain and call on it. Yep, Unix solved all your problems back in the 1970s. Pipes do not provide any fine grained control over the concurrent behavior. If you want to change the order of calls, suddenly you have to write a bash script (with its own set of issues), etc. Unix solved these problems in much the same way that the evolution of appendages solved the problem of changing location from one point to another before the automobile. The process matters. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
Do you think we'll always have a huge number of incompatible programming languages? I agree with you that it's a fact of life in 2012, but will it be a fact of life in 2062? It will be a fact of life wherever Godels theorem is; which put simplistically is: consistency and completeness cannot coexist. This is the 'logic-generator' for the mess in programming languages. Put in more general terms: Completeness is an 'adding' process Consistency is a 'subtracting' process Running the two together, convergence is hopeless. This isn't exactly correct. The incompleteness theorem basically shows that in a sufficiently expressive system, there are statements in the system that cannot be proven given the language of that system. We live with this already given the fact that the incompleteness theorem is closely related to Turing's halting problem. That doesn't indicate anything about the convergence of programming languages. It does say that we will need some form of unit testing or restricted subset simulation system as an adjunct to formal verification in most cases, until the end of time. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
MOAR TROLLING... In my opinion, people who make statements such as #1/2 are imperative, #3 is OO are missing pretty much the entire point of what OO is. OO is much more about semantics and the way code is structured. The difference between #1/2 (especially #1, of course) and #3 is surface-level syntax only. The idea of actions as concrete attributes of entities is probably the worst bit of object oriented programming, except perhaps inheritance. Events occur in the context of multiple entities, but they are conceptually distinct and should not be subordinated. Furthermore, if you have an explicit self or this context, your ability to override elements of an event is limited to things which are directly encapsulated in that context. Additionally, dynamic dispatch induces overhead and the possibility of action at a distance. Reflective template meta-programming and explicit attribute delegation are VASTLY superior. To rant about inheritance, outside of mathematics platonic ideals are bullshit. Natural kinds are the closest we might actually be able to come to ideals, and philosophers still debate whether atomic elements and subatomic particles should have this designation. On the macro scale, classes or types are dynamic, fuzzy unions of structure, context and intent. Making any sort of invariant categorical statement about real things is just begging to be proven wrong. Programmers would be much better off if we made assertions about relationships and structure with an explicit, contextual reference. I think this would simplify code reuse, since you would be able to compare contexts to know if program elements were compatible and programs could be assembled by the union or intersection of sets of assertions. About the strongest statement you can make along those lines is that #3 will allow you to do dynamic dispatch on the type of 'stack' while #1/2 won't, but even that isn't true of course. For instance, CLOS will let you write '(push stack item)' (which is the direct analogy in that language to #1) and do even more powerful dynamic dispatch than what a language like C++, Java, or Python will let you do. In the grand scheme of things, of course code structure and semantics are more important the surface-level syntax. If you take it as a given that structure/semantics are sound (big assumption, I know), then the next issue with regards to readability is the syntax itself. Sadly, defining sound (i.e. valid) language semantics is not terribly difficult; Turing complete systems occur spontaneously with surprising frequency in nature. The problem is that fundamentally, languages model the conceptual space at large, but language designers choose a semantic vocabulary that models their minuscule subset of that space. This semantic vocabulary is by its nature static, while the conceptual space is dynamic. We aren't going to get semantics really right until we model the dynamical meta structure of the concept space, and put abstract structure, context and intent at the fore. As for syntax, we have a lot of real domain specific languages, such as English, math and logic. They are vetted, understood and useful outside the context of programming. We should approach the discussion of language syntax from the perspective of trying to define a unified syntactical structure for real these DSLs.Ideally it would allow representation of things in a familiar way where possible, while providing an elegant mechanism for descriptions that cut across domains and eliminating redundancy/ambiguity. This is clearly possible, though a truly successful attempt would probably be a work of art for the ages. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
Just to troll the discussion a little bit more... On Sun, Mar 18, 2012 at 6:02 PM, Chris Angelico ros...@gmail.com wrote: On Mon, Mar 19, 2012 at 8:30 AM, John Ladasky lada...@my-deja.com wrote: What I would say is that, when PROGRAMMERS look at Python code for the first time, they will understand what it does more readily than they would understand other unfamiliar programming languages. That has value. This is something that's never truly defined. Everyone talks of how this language or that language is readable, but if you mean that you can look at a line of code and know what *that line* does, then Python suffers badly and assembly language wins out; but if you mean that you should be able to glance over an entire function and comprehend its algorithm, then I have yet to see any language in which it's not plainly easy to write bad code. Even with good code, anything more than trivial can't be eyeballed in that way - if you could, what would docstrings be for? I agree, docstrings/code comments are a pretty obvious indication that code (as it exists currently) fails as a human communication medium. I suppose that I could make an exception for elaboration on statement with subtle implications. This seems like something people should think more about fixing. Really, the metric MUST be Python programmers. Intuitiveness is of value, but readability among experienced programmers is far more useful. If I write a whole lot of code today, and next year I'm dead and someone else has replaced me, I frankly don't mind if he has to learn the language before he can grok my code. I _do_ mind if, even after he's learned the language, he can't figure out what my code's doing; and that's where Python's placed itself at about the right level - not so high that it's all in airy-fairy conceptual work, but not so low that it gets bogged down. There's a handful of other languages that are similarly placed, and they're the languages that I would call readable. In mathematics, when you perform global optimization you must be willing to make moves in the solution space that may result in a temporary reduction of your optimality condition. If you just perform naive gradient decent, only looking to the change that will induce the greatest immediate improvement in optimality, you will usually end up orbiting around a solution which is not globally optimal. I mention this because any readability or usability information gained using trained programmers is simultaneously measuring the readability or usability and its conformance to the programmer's cognitive model of programming. The result is local optimization around the current-paradigm minimum. This is why we have so many nearly identical curly brace C-like languages. In my opinion, language readability and usability should be determined based on the following tests: - Given users with no programming experience: -- What is the probability that they can correctly guess the result of a program, and how long does it take? -- What is the probability that they can take a program and correctly modify it to change its output to another specified output, or modify it to support another input representation, and how long does it take? - Given users with no programming experience who are provided with a controlled training period: -- What is the probability that they can produce a program that correctly implements a simple, completely described algorithm for solving a puzzle or winning a game, and how long does it take? - Given users with previous (but not extensive) programming experience who are provided with a controlled training period: -- What is the probability that they can produce a program that correctly implements a simple, partially described algorithm for solving a puzzle or winning a game, and how long does it take? -- What is the probability that they can produce a program that correctly implements a complex, completely described algorithm for solving a puzzle or winning a game, and how long does it take? It would probably also be worth examining user behavior relating to code organization and composition under a variety of circumstances. It seems likely that if proper organization and composition are intuitive, productivity would scale well with program size. Here's an analogy: One statement (aka line of code, etc) corresponds to one sentence in English. Massive one-liners are like some of the sentences in Paul's epistles; assembly language is like The cat sat on the mat. Both are valid; both are hard to read. This is one of my gripes with the dogmatic application of the break it into multiple statements mantra of Python. Not only are you forced to use generators to maintain semantic equivalence in many cases, in some cases a partial statement fragment doesn't have any intuitive meaning. The result is that readers are forced to hold the value of intermediate_variable in their head while reading another statement, then
Re: Python is readable
This is one of my gripes with the dogmatic application of the break it into multiple statements mantra of Python. I must admit I don't recognise that one, unless you're talking about not everything needs to be a one liner. ... Perhaps you could give some examples (actual or contrived) of stuff where breaking it into multiple statements is a bad thing? One example is performing a series of transformations on a collection of data, with the intent of finding an element of that collection that satisfies a particular criterion. If you separate out the individual transformations, you need to understand generators or you will waste space and perform many unnecessary calculations. If you only ever do a single transformation with a clear conceptual meaning, you could create a master transformation function, but what if you have a large number of potential permutations of that function? What if you are composing three or four functions, each of which is conditional on the data? If you extract things from a statement and assign them somewhat arbitrary names, you've just traded horizontal bloat for vertical bloat (with a net increase in volume), while forcing a reader to scan back and forth to different statements to understand what is happening. To steal a line from Einstein, Make things as simple as possible, but not simpler I agree, docstrings/code comments are a pretty obvious indication that code (as it exists currently) fails as a human communication medium. The fact that scientific journal articles start with a documentation string called an abstract does not indicate that scientific English fails as a human communication medium. Function docstrings say what the function does and how to use it without reading the code. They can be pulled out and displayed elsewhere. They also guide the reading of the code. Abstracts serve the same functions. A paper, with topic introduction, methods exposition, data/results description and discussion is a poor analog to a function. I would compare the abstract of a scientific paper to the overview section of a program's documentation. The great majority of the docstrings I see are basically signature rehashes with added type information and assertions, followed by a single sentence English gloss-over. If the code were sufficiently intuitive and expressive, that would be redundant. Of course, there will always be morbidly obese functions and coders that like to wax philosophical or give history lessons in comments. Also, because of Sphinx, it is very common in the Python community weave documents and code together in a way that is convenient for authors but irritating for readers. I personally would prefer not to have to scroll past 100 lines of a tutorial with examples, tests and what not in order to go from one function to another. It would be really awesome if everyone used links to that material in docstrings, and the default sphinx theme created an inline collapsible iframe that included that material for the HTML version. Don't get me wrong, I adore Sphinx, the problem here is people who are lazy or don't know the right way to structure docs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
The fact that scientific journal articles start with a documentation string called an abstract does not indicate that scientific English fails as a human communication medium. Function docstrings say what the function does and how to use it without reading the code. They can be pulled out and displayed elsewhere. They also guide the reading of the code. Abstracts serve the same functions. A paper, with topic introduction, methods exposition, data/results description and discussion is a poor analog to a function. I would compare the abstract of a scientific paper to the overview section of a program's documentation. The great majority of the docstrings I see are basically signature rehashes with added type information and assertions, followed by a single sentence English gloss-over. If the code were sufficiently intuitive and expressive, that would be redundant. Of course, there will always be morbidly obese functions and coders that like to wax philosophical or give history lessons in comments. Both abstracts and doc strings are designed to be and are read independently of the stuff they summarize. Perhaps you do not use help(obj) as often as some other people do. I find help() to be mostly useless because of the clutter induced by double under methods. I use IPython, and I typically will either use tab name completion with the ? feature or %edit obj if I really need to dig around. I teach Python to groups from time to time as part of my job, and I usually only mention help() as something of an afterthought, since typically people react to the output like a deer in headlights. Some sort of semantic function and class search from the interpreter would probably win a lot of fans, but I don't know that it is possible without a standard annotation format and the addition of a namespace cache to pyc files. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python is readable
One example is performing a series of transformations on a collection of data, with the intent of finding an element of that collection that satisfies a particular criterion. If you separate out the individual transformations, you need to understand generators or you will waste space and perform many unnecessary calculations. If you only ever do a single transformation with a clear conceptual meaning, you could create a master transformation function, but what if you have a large number of potential permutations of that function? I'm sorry, that is far too abstract for me. Do you have a *concrete* example, even an trivial one? How about a hypothetical log analyzer that parses a log file that is aggregated from multiple event sources with disparate record structures. You will need to perform a series of transformations on the data to convert record elements from text to specific formats, and your function for identifying alarm records is also dependent on record structure (and possibly system state, imagining an intrusion detection system). Of course you could go through a lot of trouble to dispatch and detect alarms over 6-7 statements, however given the description for each log record you receive, convert text elements to native data types based on the value of the first three fields of the record, then trigger an alert if that record meets defined requirements and assuming you have maps from record values to conversion functions for record elements, and a map from record types to alert criteria functions for record types already constructed, it seems like a one liner to me. What if you are composing three or four functions, each of which is conditional on the data? If you extract things from a statement and assign them somewhat arbitrary names, you've just traded horizontal bloat for vertical bloat (with a net increase in volume), while forcing a reader to scan back and forth to different statements to understand what is happening. First off, vertical bloat is easier to cope with than horizontal bloat, at least for people used to reading left-to-right rather than vertically. There are few anti-patterns worse that horizontal scrolling, especially for text. I agree that if a line goes into horizontal scroll buffer, you have a problem. Of course, I often rail on parenthesized function-taking-arguments expression structure for the fact that it forces you to read inside out and right to left, and I'd prefer not to conflate the two issues here. My assertion is that given an expression structure that reads naturally regardless, horizontal bloat is better than larger vertical bloat, in particular when the vertical bloat does not fall along clean semantic boundaries. Secondly, the human brain can only deal with a limited number of tokens at any one time. It's easier to remember large numbers when they are broken up into chunks: 824-791-259-401 versus 824791259401 (three tokens, versus twelve) Likewise for reading code. Chunking code into multiple lines instead of one long expression, and temporary variables, make things easier to understand, not harder. This is true, when the tokens are an abstraction. I read some of the research on chunking, basically it came down to people being able to remember multiple numbers efficiently in an auditory fashion using phonemes. Words versus random letter combinations have the same effect, only with visual images (which is why I think Paul Graham is full of shit with regards to his shorter is better than descriptive mantra in old essays). This doesn't really apply if storing the elements in batches doesn't provide a more efficient representation. Of course, if you can get your statements to read like sensible English sentences, there is definitely a reduction in cognitive load. And thirdly, you're not forcing the reader to scan back and forth -- or at least if you are, then you've chosen your functions badly. Functions should have descriptive names and should perform a meaningful task, not just an arbitrary collection of code. This is why I quoted Einstein. I support breaking compound logical statements down to simple statements, then combining those simple statements. The problem arises when your compound statement still looks like A B C D E F G H I J K L M N, and portions of that compound statement don't have a lot of meaning outside the larger statement. You could say X = A B C D E, Y = F G H I J, Z = K L M N, then say X Y Z, but now you've created bloat and forced the reader to backtrack. When you read: x = range(3, len(sequence), 5) you're not forced to scan back and forth between that line and the code for range and len to understand it, because range and len are good abstractions that make sensible functions. There is a lot of badly written code in existence. Don't blame poor execution of design principles on the design principle itself. I like to be fair and even handed, and I recognize that tool
Re: how to tell a method is classmethod or static method or instance method
And I'll take this opportunity to plug my dualmethod descriptor: http://code.activestate.com/recipes/577030-dualmethod-descriptor/ I use an analogous pattern in SQL Alchemy all the time (it's called hybridmethod/hybridproperty there). +1 to dualmethod, that pattern is great when you want a method or property that does something concrete when passed an instance, or something abstract relating to all instances when passed a class. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for PyPi 2.0...
Hopefully soon crate.io will be useful for finding modules ;) I have plans for it to try and, encourage people to host their code and encourage following packaging standards. I'm currently focused mostly on the backend stability (e.g. getting it stable) but emphasizing things that are generally good for the packaging ecosystem is something I hope to do. I think providing commit hooks for version control ala read the docs is the #1 thing you could do in the short term to add a lot of value. That would be enough for me to adopt the service :) Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
On Fri, Feb 10, 2012 at 5:08 AM, Chris Angelico ros...@gmail.com wrote: On Fri, Feb 10, 2012 at 1:30 PM, Nathan Rice nathan.alexander.r...@gmail.com wrote: The only thing needed to avoid the hash collision is that your hash function is not not 100% predictable just by looking at the python source code. I don't see why every dict would have to be created differently. I would think having the most ubiquitous data structure in your language be more predictable would be a priority. Oh well It's perfectly predictable. If you put a series of keys into it, you get those same keys back. Nobody ever promised anything about order. If your hash function is not 100% predictable, that means it varies on the basis of something that isn't part of either the Python interpreter or the script being run. That means that, from one execution to another of the exact same code, the results could be different. The keys will come out in different orders. I think having a hash function that is not referentially transparent is a bad thing. Basing your language on a non-deterministic function? Yeah... A type factory that produces the dict type on interpreter initialization (or, replaces the hash function, rather), and uses time/system information/etc would solve the problem, while limiting the introduced non-determinism. I don't care if the order of iteration for keys is different from interpreter run to run. I have used frozenset(mydict.items()) when my requirements dictated. It is a minor performance hit. Lets also not forget that knowing an object is immutable lets you do a lot of optimizations; it can be inlined, it is safe to convert to a contiguous block of memory and stuff in cache, etc. If you know the input to a function is guaranteed to be frozen you can just go crazy. Being able to freeze(anyobject) seems like a pretty clear win. Whether or not it is pythonic is debatable. I'd argue if the meaning of pythonic in some context is limiting, we should consider updating the term rather than being dogmatic. Just my 2 cents... Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
Lets also not forget that knowing an object is immutable lets you do a lot of optimizations; it can be inlined, it is safe to convert to a contiguous block of memory and stuff in cache, etc. If you know the input to a function is guaranteed to be frozen you can just go crazy. Being able to freeze(anyobject) seems like a pretty clear win. Whether or not it is pythonic is debatable. I'd argue if the meaning of pythonic in some context is limiting, we should consider updating the term rather than being dogmatic. Sweet, looking at the reason for rejection: 1. How dicts (and multiply nested objects) should be frozen was not completely obvious 2. frozen() implies in place, thus confusing users 3. freezing something like a list is confusing because some list methods would disappear or cause errors 4. Because automatic conversion in the proposal was seen as too involved to be so magical 5. Because frozendicts are the main end user benefit, and using dicts as keys was seen as suspect Honestly, as far as #1, we already have copy and deepcopy, the can of worms is already opened (and necessarily so). For 2, choose a better name? For 3, we have abstract base classes now, which make a nice distinction between mutable and immutable sequences; nominal types are a crutch, thinking in terms of structure is much more powerful. I agree with point 4, if magic does anything besides make the code conform to what an informed user would expect, it should be considered heretical. As for #5, I feel using a collection of key value relations as a key in another collection is not inherently bad, it just depends on the context... The mutability is the rub. Also, immutability provides scaffolding to improve performance and concurrency (both of which are top tier language features). I understand that this is one of those cases where Guido has a strong bad feeling about something, and I think a consequence of that is people tread lightly. Perhaps I'm a bit of a language communist in that regard (historically a dangerous philosophy :) As an aside, I find it kind of schizophrenic how on one hand Python is billed as a language for consenting adults (see duck typing, no data hiding, etc) and on the other hand users need to be protected from themselves. Better to serve just one flavor of kool-aid imo. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
On Thu, Feb 9, 2012 at 5:33 AM, Duncan Booth duncan.booth@invalid.invalid wrote: Nathan Rice nathan.alexander.r...@gmail.com wrote: I put dicts in sets all the time. I just tuple the items, but that means you have to re-dict it on the way out to do anything useful with it. I am too lazy to write a frozendict or import one, but I would use it if it was a builtin. I hope you sort the items before putting them in a tuple, otherwise how do you handle two identical dicts that return their items in a different order? Two dicts created from the same inputs will return items in the same arbitrary order. As long as you don't insert or delete a key you're fine. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
Two dicts created from the same inputs will return items in the same
arbitrary order. As long as you don't insert or delete a key you're
fine.
Two dicts that contain the same keys and values may or may not return them
in the same order:
dict.fromkeys('ia')
{'i': None, 'a': None}
dict.fromkeys('ai')
{'a': None, 'i': None}
Would your system count those two dicts as the same?
If the sequence in which the keys were added to the dict (and deleted if
appropriate) is exactly the same then it is likely but still not guaranteed
that they will have the same order. The only ordering guarantee is that
within a single dict keys and values will appear in a consistent order so
long as you don't add/delete keys between calls.
As I said, two dictionaries created from the same input will be the
same... 'ai' != 'ia'. If I need to hash a dict that I don't know was
created in a deterministic order, I'd frozenset(thedict.items()).
Nathan
--
http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
On Thu, Feb 9, 2012 at 11:35 AM, Ian Kelly ian.g.ke...@gmail.com wrote: On Thu, Feb 9, 2012 at 8:19 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: As I said, two dictionaries created from the same input will be the same... That's an implementation detail, not a guarantee. It will hold for current versions of CPython but not necessarily for other Python implementations. That is true. The implications of the function that creates dictionaries being non-deterministic are a little scary though, so I suspect that it will hold :) Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
On Thu, Feb 9, 2012 at 8:24 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Thu, 09 Feb 2012 09:35:52 -0700, Ian Kelly wrote: On Thu, Feb 9, 2012 at 8:19 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: As I said, two dictionaries created from the same input will be the same... That's an implementation detail, not a guarantee. It will hold for current versions of CPython but not necessarily for other Python implementations. That day may be sooner than you think. It is very likely that in Python 3.3, dict order will be randomized on creation as a side-effect of adding a random salt to hashes to prevent a serious vulnerability in dicts. http://securitytracker.com/id/1026478 http://bugs.python.org/issue13703 If there is anyone still assuming that dicts have a predictable order, they're going to be in for a nasty surprise one of these days. The only thing needed to avoid the hash collision is that your hash function is not not 100% predictable just by looking at the python source code. I don't see why every dict would have to be created differently. I would think having the most ubiquitous data structure in your language be more predictable would be a priority. Oh well Nathan -- http://mail.python.org/mailman/listinfo/python-list
Looking for PyPi 2.0...
As a user: * Finding the right module in PyPi is a pain because there is limited, low quality semantic information, and there is no code indexing. * I have to install the module to examine it; I don't need to look at docs all the time, sometimes I just want a package/class/function/method breakdown. * Given the previous point, having in-line documentation would be nice (for instance, just the output of .. automodule::) * Package usage/modification stats are not well exposed * No code metrics are available * I would like some kind of social service integration, for tagging and +1/likes. I know ratings were scrapped (and they weren't that useful anyhow), but for example, if Armin Ronacher or Robert Kern thumbs up a module there is a pretty good chance I will be interested in it. As a developer: * I don't want to have to maintain my code repository and my package releases separately. I want to let module repository know that my code repository exists, and that branches tagged as release should be made available. * I want to maintain one README. I don't like someone needs to do this now type posts but every time I use PyPi it infuratiates me. I usually end up finding modules via Stack Overflow, which seems silly to me. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: frozendict
Turn the question around: why should there be? Python is intentionally parsimonious in adding builtins. For the same reason there are frozensets? I put dicts in sets all the time. I just tuple the items, but that means you have to re-dict it on the way out to do anything useful with it. I am too lazy to write a frozendict or import one, but I would use it if it was a builtin. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: SnakeScript? (CoffeeScript for Python)
Mm I don't think it's what the OP is asking (unless I misunderstood...).
I think he wants to compile some syntax TO Python.
But I don't really see why you would something like this (if not for fun).
Maybe because you think that Python syntax could be improved upon --
for instance, Python with pattern-matching would be freaking awesome
-- but at the same time you want to leverage Python's extensive
ecosystem of libraries. So instead of creating your own brand-new
language with no third party libraries whatsoever, you create one that
just compiles down to regular Python.
You can generalize the dictionary based dispatch used for case
statements to do this. The main downsides are:
1.) You have to define your functions ahead of time or use lambdas
2.) The syntax is not quite as nice as it could be (e.g.)
foo = DispatchDict({
Pattern1: f1,
Pattern2: f2,
etc...
})
Reminds me more of javascript than I would like.
Then how are you going to maintain the code? Maintain the compiled
code or the source?
As with all compiled software, you maintain the input, not the output.
I think maintaining the output can be valuable. There are going to be
things that can be expressed in the more verbose expanded form that
will not be easily expressible in the terse pre-translated macro.
Unfortunately, most macro writers don't put much time into making sure
their macro produces concise code.
And proving that your translator is always correct
That's what unit tests are for.
I have a love hate affair with unit tests. You need them, but I'd
really rather analytically prove that my software is correct under
some set of assumptions.
Cheers,
Nathan
--
http://mail.python.org/mailman/listinfo/python-list
Constraints -//- first release -//- Flexible abstract class based validation for attributes, functions and code blocks
PyPi name: constraintslib (you'll be dissapointed if you get constraints by accident) Docs: http://packages.python.org/constraintslib/ Github: https://github.com/nathan-rice/Constraints From the docs: Constraints - Sleek contract-style validation tools === Constraints provides flexible validation tools for a variety of circumstances. All validation in constraints is done by type checking. Constraints provides a special abstract base class (Constraints) which facilitates on the fly construction of validation types. Constraints also provides a special class (Symbol) which can be used to generate natural, easy to read constraint expressions. for example:: from constraints.proxy import Symbol from constraints.constraints import Constraints X = Symbol() SizeConstraint = Constraints(X * 2 + 1 = 5) ModuloConstraint = Constraints(X % 2 != 0, X != 3) CharacterConstraint = Constraints(X[-1] == h) # My apologies for the lambda spam. I provide some functions in # constraints.util for this purpose... callable_expr = lambda x: all(lambda x: isinstance(x, SizeConstraint), x) CollectionConstraint = Constraint(callable_expr) isinstance(1, SizeConstraint) False isinstance(2, SizeConstraint) True isinstance(1, ModuloConstraint) True isinstance(blah, CharacterConstraint) True isinstance([2, 3, 4, 5], CollectionConstraint) True Constraint instances also provide descriptors which will verify values at set time. For example:: class Foo(object): ...x = Constraints(X 2) ... bar = Foo() bar.x = 1 Traceback (most recent call last): ... AssertionError: Specified value (1) does not satisfy this constraint Design by contract style preconditions, postconditions and invariants are also supported, and can be used either as context managers or function decorators:: x_pre = SizeConstraint.precondition(x) x_post = SizeConstraint.postcondition(x) x = 1 with x_pre: ... do_stuff() ... Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified pre-condition x = 5 with x_post: ... x -= 4 ... Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified post-condition @x_pre ... def foo(x): ...return x ... foo(1) Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified pre-condition @x_post ... def foo(x): ...return x - 5 ... foo(6) Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified post-condition Symbol objects are very flexible, and provide a nice way to specify your constraints without resorting to a domain specific language. Symbol objects are fairly simple; whenever an operation is performed on them, they capture it and return a new Symbol object wrapping the operation so that it can be performed with concrete input at a later time. There are exceptions to this, for example isinstance, which uses the metaclass method, and the type constructors (str, int, bool, etc) which throw an error if the correct type is not returned. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Constraints -//- first release -//- Flexible abstract class based validation for attributes, functions and code blocks
PyPi name: constraintslib (you'll be dissapointed if you get constraints by accident) Docs: http://packages.python.org/constraintslib/ Github: https://github.com/nathan-rice/Constraints From the docs: Constraints - Sleek contract-style validation tools === Constraints provides flexible validation tools for a variety of circumstances. All validation in constraints is done by type checking. Constraints provides a special abstract base class (Constraints) which facilitates on the fly construction of validation types. Constraints also provides a special class (Symbol) which can be used to generate natural, easy to read constraint expressions. for example:: from constraints.proxy import Symbol from constraints.constraints import Constraints X = Symbol() SizeConstraint = Constraints(X * 2 + 1 = 5) ModuloConstraint = Constraints(X % 2 != 0, X != 3) CharacterConstraint = Constraints(X[-1] == h) # My apologies for the lambda spam. I provide some functions in # constraints.util for this purpose... callable_expr = lambda x: all(lambda x: isinstance(x, SizeConstraint), x) CollectionConstraint = Constraint(callable_expr) isinstance(1, SizeConstraint) False isinstance(2, SizeConstraint) True isinstance(1, ModuloConstraint) True isinstance(blah, CharacterConstraint) True isinstance([2, 3, 4, 5], CollectionConstraint) True Constraint instances also provide descriptors which will verify values at set time. For example:: class Foo(object): ...x = Constraints(X 2) ... bar = Foo() bar.x = 1 Traceback (most recent call last): ... AssertionError: Specified value (1) does not satisfy this constraint Design by contract style preconditions, postconditions and invariants are also supported, and can be used either as context managers or function decorators:: x_pre = SizeConstraint.precondition(x) x_post = SizeConstraint.postcondition(x) x = 1 with x_pre: ... do_stuff() ... Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified pre-condition x = 5 with x_post: ... x -= 4 ... Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified post-condition @x_pre ... def foo(x): ...return x ... foo(1) Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified pre-condition @x_post ... def foo(x): ...return x - 5 ... foo(6) Traceback (most recent call last): ... AssertionError: The value (1) did not meet the specified post-condition Symbol objects are very flexible, and provide a nice way to specify your constraints without resorting to a domain specific language. Symbol objects are fairly simple; whenever an operation is performed on them, they capture it and return a new Symbol object wrapping the operation so that it can be performed with concrete input at a later time. There are exceptions to this, for example isinstance, which uses the metaclass method, and the type constructors (str, int, bool, etc) which throw an error if the correct type is not returned. -- http://mail.python.org/mailman/listinfo/python-list
Re: Constraints -//- first release -//- Flexible abstract class based validation for attributes, functions and code blocks
On Thu, Jan 26, 2012 at 2:51 PM, Devin Jeanpierre jeanpierr...@gmail.com wrote: Ooh, runtime turing-complete dependent-types. :) I'm not sure if you're aware of the literature on this sort of thing. It's nice reading. A library such as this that's designed for it could be used for static checks as well. Actually, that is kind of the direction I was going :) One of the nice things about Haskell is that the language is designed in a way that is conducive to proving things about your code. A side benefit of being able to prove things about your code is that in some cases you will be able to derive code just from well crafted specifications (like higher order Prolog). This isn't a game changer yet, but with advances in theorem proving software and a thoughtful language ontology, I could see it taking off very soon. Dijkstra was focused primarily on this area for the last 25 years of his life. Probably deserves a better name than constraintslib, that makes one think of constraint satisfaction. As you can probably tell from my other projects, I'm bad at coming up with snappy names. Any way to get them to raise a different error, such as ValueError (in particular for preconditions)? Currently, no. I would like to add an event mechanism, or some kind of function hooks (ala Enthought Traits but much lighter). I'm sure I'll come up with something soon :) Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Constraints -//- first release -//- Flexible abstract class based validation for attributes, functions and code blocks
May I suggest a look at languages such as ATS and Epigram? They use types that constrain values specifically to prove things about your program. Haskell is a step, but as far as proving goes, it's less powerful than it could be. ATS allows you to, at compile-time, declare that isinstance(x, 0 = Symbol() len(L)) for some list L. So it might align well with your ideas. Thanks for the tip. Probably deserves a better name than constraintslib, that makes one think of constraint satisfaction. As you can probably tell from my other projects, I'm bad at coming up with snappy names. I'm bad at doing research on previous projects ;) I guess I'm not plugging my other projects enough... You should check out elementwise. Thanks, Nathan -- http://mail.python.org/mailman/listinfo/python-list
Elementwise 0.120116 -//- beta release -//- Lazily compute functions, method calls and operations on all elements of an iterable (or graph).
Elementwise provides helpful proxy objects which let you perform a series of computations on every element of an iterable or graph, in a lazy manner. Docs: http://packages.python.org/elementwise/ GitHub: https://github.com/nathan-rice/Elementwise Examples: The standard ElementwiseProxy: nums = ElementwiseProxy([1, 2, 3, 4) print nums.bit_length() 1, 2, 2, 3 nums + 1 2, 3, 4, 5 print nums * 2 2, 4, 6, 8 print nums == 2 False, True, False, False print ((nums + 1) * 2 + 3).apply(float) 7.0, 9.0, 11.0, 13.0 print (nums.apply(float) + 0.0001).apply(round, 2) 1.0, 2.0, 3.0, 4.0 print abs(nums - 3) 2, 1, 0, 1 print (nums * 2 + 3) / 4 print efoo2.undo(3) 1, 2, 3, 4 print ((nums * 2 + 3) / 4).replicate([2, 2, 3, 3]) 1, 1, 2, 2 words = ElementwiseProxy([one, two, three, four]) print (words + little indians).upper().split( ).apply(reversed).apply(_.join) * 2 'INDIANS_LITTLE_ONEINDIANS_LITTLE_ONE', 'INDIANS_LITTLE_TWOINDIANS_LITTLE_TWO', 'INDIANS_LITTLE_THREEINDIANS_LITTLE_THREE', 'INDIANS_LITTLE_FOURINDIANS_LITTLE_FOUR' The PairwiseProxy: nums = PairwiseProxy([1, 2, 3, 4]) nums + [1, 2, 3, 4] 2, 4, 6, 8 nums * [2, 2, 3, 3] 2, 4, 9, 12 nums == [2, 2, 3, 5] False, True, True, False (nums.apply(float) / itertools.count(2) + itertools.count(1)).apply(round, args=itertools.repeat([2])) 1.5, 2.67, 3.75, 4.8 abs(nums - [3, 2, 1, 1]) 2, 0, 2, 3 (nums * [2, 2, 1, 5] + [3, 5, 9, 0]) / [4, 1, 2, 3] 1, 9, 6, 6 ((nums * itertools.repeat(2) + itertools.repeat(3)) / itertools.repeat(4)).replicate([2, 2, 3, 3]) 1, 0, 0, 0 ((nums * [2, 3, 4, 5]) [5, 6, 7, 8]) != [True, True, False, True] True, True, True, False The RecursiveElementwiseProxy: treenums = RecursiveElementwiseProxy([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) treenums + 1 ((2, 3, 4), (5, 6, 7), (8, 9, 10)) treenums * 2 ((2, 4, 6), (8, 10, 12), (14, 16, 18)) (treenums * 2 + 1).apply(float) ((3.0, 5.0, 7.0), (9.0, 11.0, 13.0), (15.0, 17.0, 19.0)) Feedback is welcome. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Elementwise 0.120116 -//- beta release -//- Lazily compute functions, method calls and operations on all elements of an iterable (or graph).
Elementwise provides helpful proxy objects which let you perform a series of computations on every element of an iterable or graph, in a lazy manner. Docs: http://packages.python.org/elementwise/ GitHub: https://github.com/nathan-rice/Elementwise Examples: The standard ElementwiseProxy: nums = ElementwiseProxy([1, 2, 3, 4) print nums.bit_length() 1, 2, 2, 3 nums + 1 2, 3, 4, 5 print nums * 2 2, 4, 6, 8 print nums == 2 False, True, False, False print ((nums + 1) * 2 + 3).apply(float) 7.0, 9.0, 11.0, 13.0 print (nums.apply(float) + 0.0001).apply(round, 2) 1.0, 2.0, 3.0, 4.0 print abs(nums - 3) 2, 1, 0, 1 print (nums * 2 + 3) / 4 print efoo2.undo(3) 1, 2, 3, 4 print ((nums * 2 + 3) / 4).replicate([2, 2, 3, 3]) 1, 1, 2, 2 words = ElementwiseProxy([one, two, three, four]) print (words + little indians).upper().split( ).apply(reversed).apply(_.join) * 2 'INDIANS_LITTLE_ONEINDIANS_LITTLE_ONE', 'INDIANS_LITTLE_TWOINDIANS_LITTLE_TWO', 'INDIANS_LITTLE_THREEINDIANS_LITTLE_THREE', 'INDIANS_LITTLE_FOURINDIANS_LITTLE_FOUR' The PairwiseProxy: nums = PairwiseProxy([1, 2, 3, 4]) nums + [1, 2, 3, 4] 2, 4, 6, 8 nums * [2, 2, 3, 3] 2, 4, 9, 12 nums == [2, 2, 3, 5] False, True, True, False (nums.apply(float) / itertools.count(2) + itertools.count(1)).apply(round, args=itertools.repeat([2])) 1.5, 2.67, 3.75, 4.8 abs(nums - [3, 2, 1, 1]) 2, 0, 2, 3 (nums * [2, 2, 1, 5] + [3, 5, 9, 0]) / [4, 1, 2, 3] 1, 9, 6, 6 ((nums * itertools.repeat(2) + itertools.repeat(3)) / itertools.repeat(4)).replicate([2, 2, 3, 3]) 1, 0, 0, 0 ((nums * [2, 3, 4, 5]) [5, 6, 7, 8]) != [True, True, False, True] True, True, True, False The RecursiveElementwiseProxy: treenums = RecursiveElementwiseProxy([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) treenums + 1 ((2, 3, 4), (5, 6, 7), (8, 9, 10)) treenums * 2 ((2, 4, 6), (8, 10, 12), (14, 16, 18)) (treenums * 2 + 1).apply(float) ((3.0, 5.0, 7.0), (9.0, 11.0, 13.0), (15.0, 17.0, 19.0)) Feedback is welcome. -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] Symbolic expressions (or: partials and closures from the inside out)
On Fri, Jan 13, 2012 at 8:45 AM, Devin Jeanpierre jeanpierr...@gmail.com wrote: On Thu, Jan 12, 2012 at 3:45 PM, Nathan Rice nathan.alexander.r...@gmail.com wrote: I'm interested in fixing both issues. I believe both issues I've had could be solved by having a robust symbolic object. These objects would basically usable like ordinary objects, however upon any attribute access or other form of interaction, the object would basically short circuit the calling function, and return a symbolic object directly to the outer scope. The symbolic object would behave like a generator function frozen at the point of attribute access, and upon send()-ing (or whatever method), it would behave exactly as if the values sent had been the ones passed in originally (ideally without consuming the generator). I find the way you've formalized this a bit weird. It looks like you're suggesting adding laziness to Python. If that's what you want, maybe you should try PyPy and the thunk object space: While thunk is neat, it doesn't accomplish precisely what I'm describing in this instance. When a function starts to run under thunk, the computations take place as soon as the function gets to the object inside its scope. What I'm after is the ability to basically create functions using only expressions, in a generative manner. Lambda does accomplish this, but in an extremely clunky manner... for example: X = lambda x: x + 1 Y = lambda y: y * 2 Z = lambda z: z % 3 (or XYZ = lambda x: (((x + 1) * 2) % 3) If I want to perform a second step after this, I need create another lambda, because they don't chain/aren't generative. The thing that Elementwise does that is very appealing is that most operations are generative, so if you have an ElementwiseProxy object x, x2 = (((x + 1) * 2) % 3) basically does the same thing as the above, but you can then do x3 = x2 ** 3, and so on until you are ready to get your results. Once you have your results, you can use the same ElementwiseProxy again with different inputs. It is just a function generation technique which has some elegant properties. Having a native object type designed to facilitate this would let people do a lot of interesting things, and make things like Elementwise be a lot more consistent. I bet you could do some really neat things by subclassing such an object as well. This is certainly not a common programming paradigm, but I think most people could reap a lot of benefits from it while being oblivious to its presence. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lib for creating Database tables
On Wed, Jan 11, 2012 at 4:38 PM, Emeka emekami...@gmail.com wrote: Hello All, I just made something pretty simple that I intend to use while creating database tables. It is still in the basic form, and much needs to be added. However, I use introspection to make it a bit easier and less work on the user. I would want my code to be reviewed by this great group. I look forward to your feedback and comments. https://github.com/janus/cheeta Having done a LOT of work with SQL Alchemy, I would definitely encourage you to just bite the bullet and get on board with it now. I routinely do very challenging things with it, and it has yet to be a roadblock. The learning curve can be somewhat steep but it is absolutely worth it. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Symbolic expressions (or: partials and closures from the inside out)
Greetings, I have been writing a lot of code lately that involves creating symbolic expressions of one form or another, which are then fully evaluated at a later time. Examples of this include Elementwise, where I create expressions that act on every member of an iterable (there is a much improved new version coming soon, by the way), and a design by contract/validation lib I'm working on (which shall remain nameless :D) that uses symbolic expressions in the *args of the metaclass __new__ method to generate a constraint class which validates input using __instancecheck__. I do most of this with lambdas, a little hacking with closures and FunctionType(), and chainable objects. I am very impressed that python is this flexible, but there are some issues with the approach that I would like to rectify, namely: 1. Because of the early binding behavior of most things in Python, if I want to include isinstance(X, someclass) in a symbolic expression, I have to wrap it in a lambda (or use .apply(), in the case of Elementwise). This is not a huge deal for me, but it forces me to create wrappers for lots of functions (e.g. isinstance_(X, someclass)) and/or have users wrap every such function they want to use in a symbolic expression. Having to do this also bloats the code a lot; the github version of Elementwise is over 3,000 LoC at this point (including prodigious documentation, but still...). 2. Python expects that certain functions, such as int(), str(), etc, will have a specific return type. While in general I agree with this, it makes Elementwise somewhat inconsistent (and it will do the same to anything else that wants to work with symbolic expressions). I'm interested in fixing both issues. I believe both issues I've had could be solved by having a robust symbolic object. These objects would basically usable like ordinary objects, however upon any attribute access or other form of interaction, the object would basically short circuit the calling function, and return a symbolic object directly to the outer scope. The symbolic object would behave like a generator function frozen at the point of attribute access, and upon send()-ing (or whatever method), it would behave exactly as if the values sent had been the ones passed in originally (ideally without consuming the generator). I have thought about ways to approximate this behavior python currently, and while I could hack something together using inspect to pull relevant info from the stack, then break out using a special exception (potentially passing a generator with state as an exception arg), this approach strikes me as VERY brittle, implementation dependent, ugly and difficult to work with. Additionally, you would need to catch the special exception somewhere in the stack, so this trick wouldn't work on the first thing in an expression to be evaluated. As an aside, I'd like to solicit some feedback on the validation syntax I've been working on. Currently, I have code that support things like: X = SymbolicObject() const = Constraints(X * 2 + 1 = 5, X % 2 != 0) const2 = Constraints(X[-1] == h) const3 = Constraints(X[-1].upper() == H) print isinstance(3, const) True print isinstance(2, const) False print isinstance(1, const) False print isinstance(bleh, const2) True print isinstance(bleh, const3) True Callables are supported as well, so if you wanted to do something like: Constraints(isinstance(X.attr, someclass), somefunc(X[-2].attr, args)) You could approximate that with: Constraints(lambda x: isinstance(x.attr, someclass), lambda x: somefunc(x[-2].attr, args)) As I mentioned in the first paragraph, Constraints is a metaclass, so your validations are checked using __instancecheck__. I'm also considering having __init__ generate mock objects (for certain straight-forward cases, anyhow). Thanks for your time, Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: [Python-ideas] Symbolic expressions (or: partials and closures from the inside out)
I have been writing a lot of code lately that involves creating
symbolic expressions of one form or another, which are then fully
evaluated at a later time. Examples of this include Elementwise,
Python is designed for concrete, rather than symbolic computation. But the
latter has been done.
Being able to create abstract expressions that are later realized is
super useful and neat. You can do this decently right now, but life
would be better if you didn't have to jump through so many hoops.
Having symbolic variables override *anything* would also make lambdas
obsolete and greatly increase the potential for lazy evaluation.
0: rewrite the text -- a bit awkward in Python.
action = compile(read({this}).format(this=book), 'xxx', 'eval')
Yeah, that is something I'd expect to see in Perl code :)
1. default args -- has to be done when the function is defined.
def action(this = book): read(this)
2. closures (nested functions) -- also requires a planned-ahead definition.
make_read(x):
return lambda: read(x)
action = make_read(book)
I use this extensively in Elementwise.
3. bound methods -- only works for classes with methods.
Class Book:
def read(self): pass
action = Book(book).read
4. partial binding of function params -- generalizes bound methods; works
for any function and argument.
from functools import partial
action = partial(read, book)
if I want to include isinstance(X, someclass) in a symbolic expression,
Consider using partial, which can totally bind all needed args *now* for
*later* action.
The issue isn't so much that I *can't* do things as they are more
trouble than they should be, and it makes the end user interface for
the stuff I write less elegant.
For example, if I want to work with a symbolic object, but include a
function that is not well behaved, or if I was creating a constraint
on the result of a function applied to a symbolic object, I have to
know ahead of time everything I want to do, so I can wrap the whole
expression in a lambda. Once I wrap it, the nice generative chaining
property disappears and I'm stuck with a callable.
from functools import partial
t = partial(isinstance, 1, int)
t()
True
f = partial(isinstance, 1, float)
f()
False
I have to wrap it in a lambda
Are you sure that partial will not work for you? Since partial is written in
Python, you can grab and adjust the code to your needs. It amounts to adding
default args after the fact by using a generic closure.
In some cases it would, in some cases it wouldn't. Since I basically
never do */** expansion on wrappers, lambdas tend to be my go-to more
often.
(or use .apply(), in the case of
Elementwise). This is not a huge deal for me, but it forces me to
create wrappers for lots of functions (e.g. isinstance_(X, someclass))
and/or have users wrap every such function they want to use in a
symbolic expression. Having to do this also bloats the code a lot;
the github version of Elementwise is over 3,000 LoC at this point
(including prodigious documentation, but still...).
2. Python expects that certain functions, such as int(), str(), etc,
will have a specific return type. While in general I agree with this,
People expect that class constructors produce an instance of the class. It
is surprising when one does otherwise ;-). Builtin classes like int, bool,
and str are classes just like ones you write.
type/str/int/etc as types is definitely semi-coherent, since the
language really treats them more like functions. They are treated
that way all over the docs as well.
From the data model page:
object.__str__(self)
Called by the str() built-in function and by the...
object.__nonzero__(self)
Called to implement truth value testing and the built-in operation bool()
object.__complex__(self)
object.__int__(self)
object.__long__(self)
object.__float__(self)
Called to implement the built-in functions complex(), int(), long(),
and float(). Should return a value of the appropriate type.
So clearly this is an area that needs some polish :)
X = SymbolicObject()
const = Constraints(X * 2 + 1= 5, X % 2 != 0)
const2 = Constraints(X[-1] == h)
const3 = Constraints(X[-1].upper() == H)
Using a special class is a standard way to delay concrete execution.
Standard, and currently a pain in the butt, starting from the fact
that operators don't hook into __getattribute__ and getting
progressively worse from there.
print isinstance(3, const)
True
A Contraints instance defines a set. 'const' is the set 'odd_ge_3'
It would look better if you used standard syntax and do the inclusion check
in a __contains__ method.
Used standard syntax? Can you elaborate please?
Also, a set is one of many things a Constraints instance could
logically be represented with, as well as a discontinuous interval, a
class in the colloquial sense, etc. The nice thing about
__instancecheck__ is that every possible constraint reduces to a type
check. Of course, you
Re: python philosophical question - strong vs duck typing
On Tue, Jan 3, 2012 at 1:13 PM, Sean Wolfe ether@gmail.com wrote: Hello everybody, I'm a happy pythonista newly subscribed to the group. How is it going? I have a theoretical / philosophical question regarding strong vs duck typing in Python. Let's say we wanted to type strongly in Python and were willing to compromise our code to the extent necessary, eg not changing variable types or casting or whatever. Let's say there was a methodology in Python to declare variable types. Do you think everyone who uses the code you write wants to deal with your type decisions? The question is, given this possibility, would this get us closer to being able to compile down to a language like C or C++? Declaring types would enable some additional optimizations, yes. No, it isn't worth it. What I am driving at is, if we are coding in python but looking for more performance, what if we had an option to 1) restrict ourselves somewhat by using strong typing to 2) make it easy to compile or convert down to C++ and thereby gain more performance. Take a look at PyPy, with RPython. That is the most future proof, forward thinking way of doing what you want. It seems to be that accepting the restrictions of strong typing might be worth it in certain circumstances. Basically the option to use a strongly-typed Python as desired. Does this get us closer to being able to convert to Cpp? Does the Cython project have anything to do with this? Declared typing is mostly annoying. Implicit static typing is less annoying, but still has issues. Cython fills the same niche as PyPy's Rpython. Use it if you have a lot of C code you want to call, as you will get better performance than a wrapper like SWIG. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Py-dea: Streamline string literals now!
Quotes are obnoxious in the nesting sense because everyone uses quotes for string delimiters. By the same token, quotes are wonderful because not only are they intuitive to programmers, but they are intuitive in general. Parenthesis are pretty much in the same boat... I *HATE* them nested, but they are so intuitive that replacing them is a non starter; Just write code that doesn't nest parenthesis. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Py-dea: Streamline string literals now!
On Wed, Dec 28, 2011 at 4:42 PM, Chris Angelico ros...@gmail.com wrote: On Thu, Dec 29, 2011 at 8:24 AM, Nathan Rice nathan.alexander.r...@gmail.com wrote: Quotes are obnoxious in the nesting sense because everyone uses quotes for string delimiters. By the same token, quotes are wonderful because not only are they intuitive to programmers, but they are intuitive in general. Parenthesis are pretty much in the same boat... I *HATE* them nested, but they are so intuitive that replacing them is a non starter; Just write code that doesn't nest parenthesis. Parentheses have different starting and ending delimiters and must be 'properly nested' (ie there must be exactly-matching inner parens inside any given set of outer parens (note that English has similar rules - you can't mis-nest parentheses (at any depth) in either language)). You can't guarantee the same about quoted strings - suppose the starting delimiter were ' and the ending (or vice versa), it still wouldn't deal with the issue of coming across an apostrophe inside a quoted string. I think you read more into my statement than was intended. Parens are bad like nested quotes are bad in the sense that they made statements difficult to read and confusing. While it is entirely possible to parse nested strings automatically in a probabilistic manner with nearly flawless accuracy by examining everything between the start and end of the line, generally I feel that people are uncomfortable with probabilistic techniques in the realm of programming :) Best just to make the user be explicit. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Python education survey
On Mon, Dec 26, 2011 at 9:52 AM, Rick Johnson rantingrickjohn...@gmail.com wrote: On Dec 25, 9:27 pm, Chris Angelico ros...@gmail.com wrote: On Mon, Dec 26, 2011 at 4:44 AM, Rick Johnson [...] Conversely, why write an IDE into IDLE when perfectly-good IDEs already exist? I don't use IDLE for development per se; it's for interactive Python execution, but not editing of source files. I believe the answer is two fold: 1. Including an IDE like IDLE into the Python distro helps noobs to get started quickly without needing to traverse a gauntlet of unknown IDEs on their own. If later they find something that they feel is more appropriate; so be it. 2. (and most important to me... IDLE is written in Python using the Tkinter GUI (which ships with python also). Therefore, the source code for IDLE can be a GREAT teaching resource for how to write professional Tkinter applications. I KNOW THE CURRENT SOURCE SUCKS! However, we could change that. So, with those points being covered, i believe IDLE is very important to the Python community and could be useful to more people IF we clean it up a bit. It's really a great little IDE with even greater potential. If Guido would just say something (or at least some of the top Pythionistas (Hettinger i am looking at you!)) this community might work together to fix this problem. Not everyone who has Python installed wants to learn the language. I do think that a learning distro that has a lot of core tools pre-installed, and ships with some tutorials, would be a decent idea. Sort of like Enthought for new users :) I don't feel IDLE is worth salvaging though. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
Elementwise provides a proxy object for iterables which supports chained method calls, as well as elementwise expressions and some built-in functions. Example: class ExampleList(ElementwiseProxyMixin, list): def __new__(cls, iterable): return list.__new__(cls, iterable) foo = ExampleList([1, 2, 3, 4]) # You could also do: efoo = ElementwiseProxy(foo) efoo = foo.each assert list(efoo.bit_length()) == [1, 2, 2, 3] print bit length: , list(efoo.bit_length()) assert list(efoo + 1) == [2, 3, 4, 5] print with addition of 1: , list(efoo + 1) assert list(efoo * 2) == [2, 4, 6, 8] print with multiplication by 2: , list(efoo * 2) assert list(efoo == 2) == [False, True, False, False] print testing equality: , efoo == 2 assert list((efoo + 1) * 2 + 3) == [7, 9, 11, 13] print chaining addition and multiplication: , (efoo + 1) * 2 + 3 Each ElementwiseProxy also has a parent attribute so you can backtrack in the chain as needed rather than store each intermediate value, if you think you might need them. There are still some issues with proper support of things like bool() and int(), which refuse to return things that are not of the correct type. Get it: PyPi: http://pypi.python.org/pypi/elementwise/0.111220 GitHub: https://github.com/nathan-rice/Elementwise This was developed as a proof of concept for expanding the role of element-wise syntax in python, and to that end I welcome comments. Nathan Rice -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Tue, Dec 20, 2011 at 8:37 PM, Joshua Landau joshua.landau...@gmail.com wrote: On 21 December 2011 00:24, Nathan Rice nathan.alexander.r...@gmail.com wrote: efoo_res = ((efoo2.capitalize() + little indian).split( ).apply(reversed) * 2).apply(_.join) # note that you could do reversed(...) instead, I just like to read left to right efoo_res.parent.parent.parent # same as ((efoo2.capitalize() + little indian).split( ) in case you need to debug something and want to look at intermediate values How is any of this better than the elementwise operators (~)? People should be able to expect len(x) to always return a number or raise an error. I know it's not part of the spec, but a lot breaks without these guarantees. When str(x) isn't a string, all the formatting code breaks*. And when the other suggestion (~str(x) or str~(x) or something similar) has all the benifits and none of the drawbacks, why should I use this? len() will always return a number or raise an error, just like the type functions (bool/int/etc) return that type or raise an error. The interpreter guarantees that for you. This has a couple of advantages over element-wise operators: 1. Because everything is handled in terms of generator chains, all operations on an ElementwiseProxy are evaluated lazily. With element-wise operator overloading you would need to perform each operation immediately. 2. As a result of #1, you can undo operations you perform on an ElementwiseProxy with the parent property. 3. This still works if the person who created the class you're working with doesn't add support for element-wise operators. Sure, you could monkey patch their code, but that can lead to other problems down the line. 4. There isn't an obvious/intuitive character for element-wise versions of operators, and fewer symbols is better than more IMHO (see: Perl). Also, if you use the ElementwiseProxyMixin, you can sprinkle element-wise stuff in neatly just by using variable.each where you would use ~ in your examples. Upon this implementation I take back my comment on the whole typing thing. Your title just really confused me. * You can't just make functions non-elementwise unless called through .apply either: def str(x): return x.__str__() str is one of the special cases (along with repr, unicode, int, float, long, bool, etc). These can't ever be elementwise in CPython (I don't know that this holds in other interpreters). The idea is that you use ElementwiseProxy(x) or preferably x.each, work with your data in an element-wise capacity as needed, then list() or iter() back out. I think it should be a deliberate change of context. Interleaving the proxy (say via x.each) in mostly scalar expressions works, but wasn't the use case I designed it for, and in those circumstances it doesn't really offer anything above the map function. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
Have you seen PyLINQ? It has a similar approach to operating on
collections, returning a PyLINQ object after each call to facilitate
chaining. https://github.com/kalessin/PyLINQ/blob/master/pylinq/linq.py
I hadn't seen it previously. I am a VERY heavy user of SQL Alchemy
though, and I am sure chaining generative ClauseElements/Queries/etc,
has burned some patterns into my subconscious at this point.
This is a personal opinion on the code, but I'd move instantiating the
new ElementwiseProxy out of each method and into its own decorator:
# declare this outside of the class
def chainable(fn):
def _(self, *args, **kwargs):
return ElementwiseProxy(fn(self, *args, **kwargs), self)
return _
This way, each method that is chainable is a little more obvious
without inspecting the code, and the method body itself is only doing
what the method says it does:
@chainable
def __add__(self, other):
return (e + other for e in object.__getattribute__(self,
iterable))
This is a reasonable suggestion and I will play with something along
those lines soon.
Incidentally, displaying an ElementwiseProxy instance doesn't go down
well with iPython:
In [1]: from elementwise import *
In [2]: e = ElementwiseProxy(['one','two','three'])
In [3]: e
Out[3]: ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (6, 0))
I love IPython, but it has had known problems with iterators for
years. A long time ago, I agonized over what I thought was a bug in
my code where itertools.count would skip numbers in IPython, but my
unit tests all passed. Everything should work fine if you tuple() it
first.
Nathan
--
http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Wed, Dec 21, 2011 at 11:29 AM, Robert Kern robert.k...@gmail.com wrote:
On 12/21/11 3:15 PM, Nathan Rice wrote:
Incidentally, displaying an ElementwiseProxy instance doesn't go down
well with iPython:
In [1]: from elementwise import *
In [2]: e = ElementwiseProxy(['one','two','three'])
In [3]: e
Out[3]: ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (6, 0))
I love IPython, but it has had known problems with iterators for
years. A long time ago, I agonized over what I thought was a bug in
my code where itertools.count would skip numbers in IPython, but my
unit tests all passed. Everything should work fine if you tuple() it
first.
This is a different problem, actually. The problem is that the recently
added (by me, actually) pretty-printing system tries to dispatch based on
the type. In order to handle old-style classes, it checks for the type using
.__class__ first. ElementProxy's __getattribute__() gets in the way here by
returning a generator instead of the ElementProxy class.
Thanks for the heads up Robert. Given that IPython is awesome and
people should probably be doing an '.apply(type)' in that instance
anyhow, I will corner case that on __getattribute__.
Nathan
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python education survey
+1 for IPython/%edit using the simplest editor that supports syntax highlighting and line numbers. I have found that Exploring/Prototyping in the interpreter has the highest ROI of anything I teach people. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Wed, Dec 21, 2011 at 12:07 PM, Paul Dubois pfdub...@gmail.com wrote: You're reinventing Numeric Python. I prefer to think of it in terms of paying homage to NumPy (and functional programming). A big part of the motivation for this was bringing the elegance of NumPy to other areas of python besides numerical/scientific programming. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Tue, Dec 20, 2011 at 8:37 PM, Joshua Landau joshua.landau...@gmail.com wrote: On 21 December 2011 00:24, Nathan Rice nathan.alexander.r...@gmail.com wrote: efoo_res = ((efoo2.capitalize() + little indian).split( ).apply(reversed) * 2).apply(_.join) # note that you could do reversed(...) instead, I just like to read left to right efoo_res.parent.parent.parent # same as ((efoo2.capitalize() + little indian).split( ) in case you need to debug something and want to look at intermediate values How is any of this better than the elementwise operators (~)? People should be able to expect len(x) to always return a number or raise an error. I know it's not part of the spec, but a lot breaks without these guarantees. When str(x) isn't a string, all the formatting code breaks*. And when the other suggestion (~str(x) or str~(x) or something similar) has all the benifits and none of the drawbacks, why should I use this? len() will always return a number or raise an error, just like the type functions (bool/int/etc) return that type or raise an error. The interpreter guarantees that for you. The point wasn't that either way was better, but that with this implementation you get neither choice (len(x) vs len~(x)) or reliability. If len didn't have the hard coded behavior, you would have the choice of len(x) or len(x.each). Since a lot of code relies on len() returning an it, I think it is fine to accept that you have to use x.each.apply(len). I agree that this is a case where some kind of elementwise designation in the syntax would be better; if there was a ∀ character on the keyboard and more people knew what it meant I would have fewer reservations. The reliability point works like this: You want to elementwise a few functions, that before you were doing on a single item. BEFORE: item = foreignfunc1(item) item = foreignfunc2(item) item = foreignfunc3(item) NOW (your method): item = ElementwiseProxy(item) item = foreignfunc1(item) item = foreignfunc2(item) item = foreignfunc3(item) item = list(item) well, I would say it more like: item.each.apply(foreignfunc1).apply(foreignfunc2).apply(foreignfunc3) # I like to read left to right, what can I say? and I wouldn't list() it right away, since it is nice and lazy. You might think your method works. But what if foreignfunc is str? And you can't blacklist functions. You can't say everything works bar A, B and C. What if you get: lambda x:str(x)? You can't blacklist that. That makes the ElementwiseProxy version buggy and prone to unstable operation. If it's consistent, fine. But it's not. I don't need to blacklist anything. Everything that has funny behavior like str goes through special methods on the class (that I know of), and isn't hooked through __getattribute__, so I just handle it somewhat normally. 1. Because everything is handled in terms of generator chains, all operations on an ElementwiseProxy are evaluated lazily. With element-wise operator overloading you would need to perform each operation immediately. I agree this can be a preferred advantage.But you still have to choose one. ~ could be lazy, or it could be eager. But in both implementations you have to choose. That said, you have map and imap, and so you could have ElemetwiseProxy and eElementwiseProxy (eager), and you can have ~ and i~. Remember that the syntax I'm using is just for PEP consistency. Some more creative people can find a syntax that works. From my perspective the strength of operators is that they are intuitive, since we use them constantly in other areas; I can make a good guess about what X * Y or X + Y means in various contexts. When you introduce operators that don't have any well ingrained, standard meaning, you just make the syntax cryptic. 2. As a result of #1, you can undo operations you perform on an ElementwiseProxy with the parent property. Use case? If this is actually a wanted feature, parent could be made a general property of iterators. (x for x in foo).parent == foo I think that's a separate proposal that isn't intrinsic to this idea. One quick use case: I want to debug something big, slow and nasty; . I set breakpoints with conditions where it seems like the issue lies from the stack trace. Unfortunately I missed the root cause, and the variables that would help me debug it have been garbage collected. I can go through the entire process again, try to set better breakpoints and cross my fingers, or iterate over some subset of parent operations right there. I think I'll take the latter. 3. This still works if the person who created the class you're working with doesn't add support for element-wise operators. Sure, you could monkey patch their code, but that can lead to other problems down the line. As I understood it, the elementwise operators in the PEP weren't language magic, but new hooks to special methods. If it is language magic, that will probably
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Wed, Dec 21, 2011 at 12:53 PM, Arnaud Delobelle arno...@gmail.com wrote: You can already do: efoo2 = [one, two, three, four] [_.join(reversed((x.capitalize() + little indian).split( )) * 2) for x in efoo2] Note 1: I've ignored the fact that reversed(...)*2 is erroneous Note 2: I wouldn't such code myself, in either form What's the advantage of your elementwise stuff? Sit two theoretically identical programmers who have a decent grasp of python, understand the basics of ElementwiseProxy and the components of your version down side by side, and see which one can figure out what the output should be first. I will bet real money it is the one working with the ElementwiseProxy code. We have all had a lot of practice reading inside out and backwards in the python community with comprehensions and such, but it is in no way natural or easy. Additionally, everything done via an ElementwiseProxy is lazy, so you only pay for what you eat :) Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
It doesn't seem to work correctly when both operands are Elementwise: numerator = ElementwiseProxy(range(5)) denominator = ElementwiseProxy([2, 2, 3, 3, 3]) remainder = numerator % denominator print remainder Traceback (most recent call last): File stdin, line 1, in module File elementwise.py, line 72, in __repr__ return , .join(e.__repr__() for e in object.__getattribute__(self, iterable)) TypeError: sequence item 0: expected string, ElementwiseProxy found list(remainder) [] Cheers, Ian Ian, can you clarify the expected output in that case? My initial guess would be that you want to do a modulo on the cartesian product of parameters from the proxies, sort of like: import itertools numerator = range(5) denominator = [2, 2, 3, 3, 3] remainder = [n % d for (n, d) in itertools.product(numerator, denominator)] Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Wed, Dec 21, 2011 at 1:24 PM, Joshua Landau joshua.landau...@gmail.com wrote: I was under the impression that these were meant to be interchangeable. This is because functions are just wrappers to non-functions, really. from elementwise import ElementwiseProxy as P (lambda x:x+[1])(P([[0],[0],[0]])) [0, 1], [0, 1], [0, 1] If we have to use .apply, we might as well use map :P. Apply is not required, you can use functions. I just hate reading inside out and backwards. compare: func3(func2(func1(x))) # Fine if the chain is short, but quickly gets unreadable x.apply(func1).apply(func2).apply(func3) # More verbose when working with short chains, but infinitely clearer and more readable when things start getting complicated, or for less experienced coders Note that len and dir crash. Here is a perfect example: int(P([1,2,3])) Traceback (most recent call last): File stdin, line 1, in module TypeError: __int__ returned non-int (type ElementwiseProxy) It is alpha software :P Easily fixed though. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Wed, Dec 21, 2011 at 1:41 PM, Ian Kelly ian.g.ke...@gmail.com wrote: I expected the equivalent of: remainder = [n % d for (n, d) in zip(numerator, denominator)] which is what numpy does. I do want to come up with a nice way to do that... However: if numerator and denominator are plain lists that magically have an each method... each_numerator = numerator.each each_denominator = denominator.each each_numerator % each_denominator see where I'm going with the mismatch on the semantics? I know the example is a bit contrived, sorry. Expanding iterators by default might be a worth a shot though, since passing around iterators like that is pretty rare. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: [TIP] Anyone still using Python 2.5?
Just because the default python version on a server is 2.4 doesn't mean you can't install 2.7.2... If the admins that run the machine are too lazy/stupid to install a second copy of Python let them rot. Of course, if by some nightmare scenario you have code that can't be upgraded for whatever reason, I'm so sorry. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
Elementwise provides a proxy object for iterables which supports chained method calls, as well as elementwise expressions and some built-in functions. Example: class ExampleList(ElementwiseProxyMixin, list): def __new__(cls, iterable): return list.__new__(cls, iterable) foo = ExampleList([1, 2, 3, 4]) # You could also do: efoo = ElementwiseProxy(foo) efoo = foo.each assert list(efoo.bit_length()) == [1, 2, 2, 3] print bit length: , list(efoo.bit_length()) assert list(efoo + 1) == [2, 3, 4, 5] print with addition of 1: , list(efoo + 1) assert list(efoo * 2) == [2, 4, 6, 8] print with multiplication by 2: , list(efoo * 2) assert list(efoo == 2) == [False, True, False, False] print testing equality: , efoo == 2 assert list((efoo + 1) * 2 + 3) == [7, 9, 11, 13] print chaining addition and multiplication: , (efoo + 1) * 2 + 3 Each ElementwiseProxy also has a parent attribute so you can backtrack in the chain as needed rather than store each intermediate value, if you think you might need them. There are still some issues with proper support of things like bool() and int(), which refuse to return things that are not of the correct type. Get it: PyPi: http://pypi.python.org/pypi/elementwise/0.111220 GitHub: https://github.com/nathan-rice/Elementwise This was developed as a proof of concept for expanding the role of element-wise syntax in python, and to that end I welcome comments. Nathan Rice -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
If you take a moment and examine the version number, you will notice that it is a date code. In my opinion that is far more informative than an arbitrary number. I use the major version number to signify... Wait for it... Major changes :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Tue, Dec 20, 2011 at 4:00 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Tue, 20 Dec 2011 15:45:07 -0500, Nathan Rice wrote: If you take a moment and examine the version number, you will notice that it is a date code. Not any date code I'm familiar with. 0.111220 doesn't look anything like a date to me. Possibly if the last release was two thousand years ago. I'd rather stick to actively maintained software, if it's all the same with you. Date code != date. In my opinion that is far more informative than an arbitrary number. I use the major version number to signify... Wait for it... Major changes :) Well, that's one opinion. Another opinion is that nobody cares what specific day you release a new version, and that versions 0.191231 and 0.200101 probably aren't that big a difference. Nobody cares about version numbers in general, except as a way to fulfill dependencies. By using a date code, your versions are guaranteed to sort in release order (which is nice, say if someone was to download your software via FTP), you can tell what release has what ticket fixes in an issue tracker, stuff like that. It also gives me an easy way to be nostalgic about releases As for the extra 20 that I exclude, if I haven't updated the major version number by the time 2020 rolls around I deserve any trouble it causes :) -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
On Tue, Dec 20, 2011 at 7:03 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Tue, Dec 20, 2011 at 12:45 PM, Nathan Rice nathan.alexander.r...@gmail.com wrote: There are still some issues with proper support of things like bool() and int(), which refuse to return things that are not of the correct type. And that's a good thing. As type conversion functions, bool(x) and int(x) should *always* return bools and ints respectively (or raise an exception), no matter what you pass in for x. I was hoping to have the proxy be completely transparent. I don't disagree with your statement though. If I do list(efoo), where efoo is an ElementwiseProxy object, should I expect to get the efoo collection converted to a list, or another ElementwiseProxy where each element has been converted to a list? I would hope the former. Iterators are how you go from an ElementwiseProxy back to a regular collection. Thus list/set/etc or anything that takes an iterator will work. This was developed as a proof of concept for expanding the role of element-wise syntax in python, and to that end I welcome comments. The examples you gave are all numerical in nature. If I might inquire, what might I use this for that I can't already do with numpy? efoo2 = ElementwiseProxy([one, two, three, four]) efoo_res = ((efoo2.capitalize() + little indian).split( ).apply(reversed) * 2).apply(_.join) # note that you could do reversed(...) instead, I just like to read left to right efoo_res.parent.parent.parent # same as ((efoo2.capitalize() + little indian).split( ) in case you need to debug something and want to look at intermediate values The idea is to provide a syntax that lets you do very complex things on collections in a more readable manner, without having 5 or 6 lines of generator expressions. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Elementwise -//- first release -//- Element-wise (vectorized) function, method and operator support for iterables in python.
efoo2 = ElementwiseProxy([one, two, three, four]) efoo_res = ((efoo2.capitalize() + little indian).split( ).apply(reversed) * 2).apply(_.join) # note that you could do reversed(...) instead, I just like to read left to right efoo_res.parent.parent.parent # same as ((efoo2.capitalize() + little indian).split( ) in case you need to debug something and want to look at intermediate values Just a quick note, I realized after I sent this that reversed returns an iterator, so * will not work. Otherwise everything is kosher. -- http://mail.python.org/mailman/listinfo/python-list
Re: Making the case for typed lists/iterators in python
I think there are two aspects to your idea: 1. collections that share a single type 2. accessing multiple elements via a common interface You are correct, and I now regret posing them in a coupled manner. Both are things that should be considered and I think both are useful in some contexts. The former would provide additional guarantees, for example, you could savely look up an attribute of the type only once while iterating over the sequence and use it for all elements. Also, I believe you could save some storage. The second aspect would mean that you have a single function call that targets multiple objects, which is syntactic sugar, but that's a good thing. To some extent, this looks like a C++ valarray, see e.g. [1] and [2] (note that I don't trust [1] and that [2] is perhaps a bit outdated), in case you know C++ and want to draw some inspiration from this. Anyway, I believe something like that would already be possible today, which would give people something they could actually try out instead of just musing about: class ValarrayWrapper(object): def __init__(self, elements): self._elements = elements def map(self, function): tmp = [function(x) for x in self._elements] return ValarrayWrapper(tmp) def apply(self, function) self._elements[:] = [function(x) for x in self._elements] I could even imagine this to implement generic attribute lookup by looking at the first element. If it contains the according attribute, return a proxy that allows calls to member functions or property access, depending on the type of the attribute. Thank you for the references, I am always interested to see how other languages solve problems. I have received the code please comment repeatedly, I will have to take some time after work today to deliver. I believe that typed lists that get demoted to normal lists with a warning on out of type operations preserve this information while providing complete backwards compatibility and freedom. I don't think that a warning helps people write correct code, a meaningful error does. Otherwise, with the same argument you could convert a tuple on the fly to a list when someone tries to change an element of it. I do agree errors are more normative than warnings. The problem with an error in these circumstances is it will certainly break code somewhere. Perhaps a warning that becomes an error at some point in the future would be the prudent way to go. Thanks! Nathan -- http://mail.python.org/mailman/listinfo/python-list
Making the case for typed lists/iterators in python
I realize this has been discussed in the past, I hope that I am presenting a slightly different take on the subject that will prove interesting. This is primarily motivated by my annoyance with using comprehensions in certain circumstances. Currently, if you want to perform successive transformations on the elements of a list, a couple of options: 1. Successive comprehensions: L2 = [X(e) for e in L1] L3 = [Y(e) for e in L2] L4 = [Z(e) for e in L3] or L2 = [e.X() for e in L1] This gets the job done and gives you access to all the intermediate values, but isn't very succinct, particularly if you are in the habit of using informative identifiers. 2. One comprehension: L2 = [Z(X(Y(e))) for e in L1] or L2 = [e.X().Y().Z() for e in L1] This gets the job done, but doesn't give you access to all the intermediate values, and tends to be pretty awful to read. Having typed lists let you take preexisting string/int/etc methods and expose them in a vectorized context and provides an easy way for developers to support both vectors and scalars in a single function (you could easily fix other people's functions dynamically to support both). Additionally, typed lists/iterators will allow improved code analysis and optimization. The PyPy people have already stated that they are working on implementing different strategies for lists composed of a single type, so clearly there is already community movement in this direction. Just compare the above examples to their type-aware counterparts: L2 = X(L1) L2 = L1.X() L2 = Z(Y(X(L1))) L2 = L1.X().Y().Z() Also, this would provide a way to clean up stuff like: \n.join(l.capitalize() for l in my_string.split(\n)) into: my_string.split(\n).capitalize().join_this(\n) Before anyone gets up in arms at the idea of statically typed python, what I am suggesting here would be looser than that. Basically, I believe it would be a good idea in instances where it is known that a list of single type is going to be returned, to return a list subclass (for example, StringList, IntegerList, etc). To avoid handcuffing people with types, the standard list modification methods could be hooked so that if an object of an incorrect type is placed in the list, a warning is raised and the list converts to a generic object list. The only stumbling block is that you can't use __class__ to convert from stack types to heap types in CPython. My workaround for this would be to have a factory that creates generic List classes, modifying the bases to produce the correct behavior. Then, converting from a typed list to a generic object list would just be a matter of removing a member from the bases for a class. This of course basically kills the ability to perform type specific list optimization in CPython, but that isn't necessarily true for other implementations. The additional type information would be preserved for code analysis in any case. The case would be even simpler for generators and other iterators, as you don't have to worry about mutation. I'd like to hear people's thoughts on the subject. Currently we are throwing away useful information in many cases that could be used for code analysis, optimization and simpler interfaces. I believe that typed lists that get demoted to normal lists with a warning on out of type operations preserve this information while providing complete backwards compatibility and freedom. Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Making the case for typed lists/iterators in python
Nothing stops me from implementing it, in fact it is VERY trivial to wrap member class methods onto a list subclass, and wrap functions to support vectorized behavior. The problem is that as soon as you hit anything outside your code that returns a list or iterator, everything gets blown away unless you explicitly wrap the return value, which entirely defeats the point. On Fri, Dec 16, 2011 at 1:23 PM, Stefan Behnel stefan...@behnel.de wrote: Nathan Rice, 16.12.2011 18:48: I realize this has been discussed in the past, I hope that I am presenting a slightly different take on the subject that will prove interesting. This is primarily motivated by my annoyance with using comprehensions in certain circumstances. Currently, if you want to perform successive transformations on the elements of a list, a couple of options: 1. Successive comprehensions: L2 = [X(e) for e in L1] L3 = [Y(e) for e in L2] L4 = [Z(e) for e in L3] or L2 = [e.X() for e in L1] This gets the job done and gives you access to all the intermediate values, but isn't very succinct, particularly if you are in the habit of using informative identifiers. 2. One comprehension: L2 = [Z(X(Y(e))) for e in L1] or L2 = [e.X().Y().Z() for e in L1] This gets the job done, but doesn't give you access to all the intermediate values, and tends to be pretty awful to read. Having typed lists let you take preexisting string/int/etc methods and expose them in a vectorized context and provides an easy way for developers to support both vectors and scalars in a single function (you could easily fix other people's functions dynamically to support both). Additionally, typed lists/iterators will allow improved code analysis and optimization. The PyPy people have already stated that they are working on implementing different strategies for lists composed of a single type, so clearly there is already community movement in this direction. Just compare the above examples to their type-aware counterparts: L2 = X(L1) L2 = L1.X() L2 = Z(Y(X(L1))) L2 = L1.X().Y().Z() What keeps you from implementing this? You don't need to change the language for it, just wrap the list in a class that overrides __getattr__() to return something that does the appropriate transformation for each element. I would be surprised if you needed more than a couple of lines of Python code for that. Stefan -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Odd behavior of object equality/identity in the context of relative vs fully qualified imports
I just ran into this yesterday, and I am curious if there is a rational behind it... I have a class that uses a dictionary to dispatch from other classes (k) to functions for those classes (v). I recently ran into a bug where the dictionary would report that a class which was clearly in the dictionary's keys was giving a KeyError. id() produced two distinct values, which I found to be curious, and issubclass/isinstance tests also failed. When I inspected the two classes, I found that the only difference between the two was the __module__ variable, which in one case had a name relative to the current module (foo), and in another case had the fully qualified name (bar.foo). When I went ahead and changed the import statement for the module to import bar.foo rather than import foo, everything worked as expected. My first thought was that I had another foo module in an old version of the bar package somewhere on my pythonpath; After a thorough search this proved not to be the case. Has anyone else run into this? Is this intended behavior? If so, why? Nathan -- http://mail.python.org/mailman/listinfo/python-list
Re: Odd behavior of object equality/identity in the context of relative vs fully qualified imports
It would be more work than I want to go into to provide full context (unless it is to file a bug report, if it actually is a bug). I verified that there are no cyclical dependency issues using snakefood, and I doublechecked that just changing the import from full to partial name is sufficient to reintroduce the bug. Can I get confirmation that this is not expected behavior? I will go ahead and file a bug report it that is the case. Nathan On Thu, Dec 15, 2011 at 10:08 AM, Dave Angel d...@davea.name wrote: On 12/15/2011 09:34 AM, Nathan Rice wrote: I just ran into this yesterday, and I am curious if there is a rational behind it... I have a class that uses a dictionary to dispatch from other classes (k) to functions for those classes (v). I recently ran into a bug where the dictionary would report that a class which was clearly in the dictionary's keys was giving a KeyError. id() produced two distinct values, which I found to be curious, and issubclass/isinstance tests also failed. When I inspected the two classes, I found that the only difference between the two was the __module__ variable, which in one case had a name relative to the current module (foo), and in another case had the fully qualified name (bar.foo). When I went ahead and changed the import statement for the module to import bar.foo rather than import foo, everything worked as expected. My first thought was that I had another foo module in an old version of the bar package somewhere on my pythonpath; After a thorough search this proved not to be the case. Has anyone else run into this? Is this intended behavior? If so, why? Nathan Hard to tell with such generic information. But I'm guessing you imported your script from some other module, creating a circular import sequence. The circular can be a problem in its own right. But even worse, if the script is part of the chain is that it's loaded twice, with different names. And any top-level items, such as classes, will be instantiated twice as well. is your script called foo.py by any chance? -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: unittest. customizing tstloaders / discover()
Nose is absolutely the way to go for your testing needs. You can put __test__ = False in modules or classes to stop test collection. On Mon, Dec 12, 2011 at 5:44 AM, Thomas Bach bac...@uni-mainz.de wrote: Gelonida N gelon...@gmail.com writes: Do I loose anything if using nose. or example can all unit tests / doc tests still be run from nose? AFAIK you don't loose anything by using nose – the unittests should all be found and doctests can be run via `--with-doctest', I never used doctests though. regards -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Working with databases (ODBC and ORMs) in Python 3.2
For now, get started in Python 2.7. Write code with an eye to 3.x portability, and you will be fine. You probably won't see 3.x overtake 2.x for at least 3-4 years, and a decent amount of stuff is still 2.x only. Since it sounds like you are a windows/net shop, go ahead and use Iron Python. SQL Alchemy is your one stop shop. It is basically 3.x compatible, but see my previous statement about 3.x. I haven't used SQL Alchemy with SQL Server but honestly, SQL Alchemy is THE showcase python library -- I doubt you will run into any issues. Just google sqlalchemy sql server, I'm sure there's a blog post explaining the specifics in detail. Nathan On Thu, Nov 10, 2011 at 1:56 PM, tkp...@hotmail.com tkp...@hotmail.com wrote: We are in the process of trying to decide between Python 2.7 and 3.2 with a view to making a 5-10 year commitment to the right platform, and would appreciate some guidance on how best to connect to SQL databases in 3.2. ceODBC 2.01 provides an ODBC driver for Python 3.2, does anyone have experience using it? Also, are there any ORMs (object relational mapper)s that work well with 3,2? Thanks in advance Thomas Philips -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Forking simplejson
Just a random note, I actually set about the task of re-implementing a json encoder which can be subclassed, is highly extensible, and uses (mostly) sane coding techniques (those of you who've looked at simplejson's code will know why this is a good thing). So far preliminary tests show the json only subclass of the main encoder basically tied in performance with the python implementation of simplejson. The C version of simplejson does turn in a performance about 12x faster, but that's apples to oranges. The design of the encoder would also make a XML serializer pretty straight forward to implement as well (not that I care about XML, *blech*). I'm torn between just moving on to some of my other coding tasks and putting some time into this to make it pass the simplejson/std lib json tests. I really do think the standard lib json encoder is bad and I would like to see an alternative in there but I'm hesitant to get involved. Nathan On Thu, Oct 27, 2011 at 11:24 AM, Amirouche Boubekki amirouche.boube...@gmail.com wrote: 2011/10/27 Chris Rebert c...@rebertia.com On Wed, Oct 26, 2011 at 2:14 AM, Amirouche Boubekki amirouche.boube...@gmail.com wrote: Héllo, I would like to fork simplejson [1] and implement serialization rules based on protocols instead of types [2], plus special cases for protocol free objects, that breaks compatibility. The benefit will be a better API for json serialization of custom classes and in the case of iterable it will avoid a calls like: simplejson.dumps(list(my_iterable)) The serialization of custom objects is documented in the class instead of the ``default`` function of current simplejson implementation [3]. The encoding algorithm works with a priority list that is summarized in the next table: +---+---+ | Python protocol | JSON | | or special case | | +===+===+ snip | (§) unicode | see (§) | snip (§) if the algorithm arrives here, call unicode (with proper encoding rule) on the object and use the result as json serialization I would prefer a TypeError in such cases, for the same reason str.join() doesn't do an implicit str() on its operands: - Explicit is better than implicit. - (Likely) errors should never pass silently. - In the face of ambiguity, refuse the temptation to guess. granted it's better. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Forking simplejson
I've found that in a lot of cases getting a patch submitted is only half about good engineering. The other half is politics. I like one of those things, I don't like the other, and I don't want to take time out of my coding schedule to write something if in the end a reviewer shoots down my patch for contrived reasons. I don't know what the python committers are like but I guess you could say I'm once bitten twice shy. Nathan On Fri, Oct 28, 2011 at 4:52 PM, Terry Reedy tjre...@udel.edu wrote: On 10/28/2011 1:20 PM, Nathan Rice wrote: Just a random note, I actually set about the task of re-implementing a json encoder which can be subclassed, is highly extensible, and uses (mostly) sane coding techniques (those of you who've looked at simplejson's code will know why this is a good thing). So far preliminary tests show the json only subclass of the main encoder basically tied in performance with the python implementation of simplejson. The C version of simplejson does turn in a performance about 12x faster, but that's apples to oranges. The design of the encoder would also make a XML serializer pretty straight forward to implement as well (not that I care about XML, *blech*). I'm torn between just moving on to some of my other coding tasks and putting some time into this to make it pass the simplejson/std lib json tests. I really do think the standard lib json encoder is bad Python is the result of people who thought *something* was 'bad' and I would like to see an alternative in there and volunteered the effort to make something better. but I'm hesitant to get involved. As someone who is involved and tries to encourage others, I am curious why. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Forking simplejson
Since this happily went off to the wrong recipient the first time... The python json module/simpljson are badly in need of an architecture update. The fact that you can't override the encode method of JSONEncoder and have it work reliably without monkey patching the pure python encoder is a sign that something is horribly wrong. On Wed, Oct 26, 2011 at 5:14 AM, Amirouche Boubekki amirouche.boube...@gmail.com wrote: Héllo, I would like to fork simplejson [1] and implement serialization rules based on protocols instead of types [2], plus special cases for protocol free objects, that breaks compatibility. The benefit will be a better API for json serialization of custom classes and in the case of iterable it will avoid a calls like: simplejson.dumps(list(my_iterable)) The serialization of custom objects is documented in the class instead of the ``default`` function of current simplejson implementation [3]. The encoding algorithm works with a priority list that is summarized in the next table: +---+---+ | Python protocol | JSON | | or special case | | +===+===+ | (ø) __json__ | see (ø) | +---+---| | map | object| +---+---+ | iterable | array | +---+---+ | (*) float,int,long| number| +---+---+ | (*) True | true | +---+---+ | (*) False | false | +---+---+ | (*) None | null | +---+---+ | (§) unicode | see (§) | +---+---+ (ø) if the object implements a __json__ method, the returned value is used as the serialization of the object (*) special objects which are protocol free are serialized the same way it's done currently in simplejson (§) if the algorithm arrives here, call unicode (with proper encoding rule) on the object and use the result as json serialization As soon as an object match a rule, it's serialized. What do you think ? Do you find this API an improvement over simplejson ? Is it worth to code ? Where are documented the different protocols implemented by Python objects ? Regards, Amirouche [1] https://github.com/simplejson/simplejson [2] https://github.com/simplejson/simplejson/blob/master/simplejson/encoder.py#L75 [3] http://simplejson.readthedocs.org/en/latest/index.html#simplejson.JSONEncoder.default -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Java is killing me! (AKA: Java for Pythonheads?)
SNIP
public FooClass(String requiredArgument1, Long requiredArgument2, map
yourOptionalArgumentMap) {
...
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: Struqtural: High level database interface library
Oh yes, I'd rather write lines of that rather than pages of SQL in a Python string. (not to mention, avoid some easy to fall into security flaws, not have to worry about porting dialect specific SQL code, etc, etc). Fixed that for you. I can't take the credit for that part though, that magic comes from SQL Alchemy. All Struqtural does is let you move data into a database easily, move data from a database to Python easily, and simplify the creation of some common complex schemas. -- http://mail.python.org/mailman/listinfo/python-list
[ANN] Struqtural: High level database interface library
Struqtural makes it easy to get data into a database, and easy to work
with it once it's there. Some of the big features include:
* Automatically generate all tables and relations needed to represent
XML in a database, including one to one, one to many, many to one and
many to many relationships (JSON and YAML support is planned for a
future release), returning mapped objects.
* Automatically generate python objects for a specified table in a
database, with support for discovery of all types of relationships
(sort of like SQL Soup on 4 different kinds of steroids).
* Automatically create persisted representations of python objects in
a database, along with a persistent version of the object class.
* Automatically infer SQL data types and create table representations
for delimited text files (i.e. CSV/TSV), returning mapped objects.
* Easily generate and query EAV/vertical attribute tables.
* Easily generate and query graphs/trees/directed acyclic graphs.
* Easily manage session configuration and creation for multiple databases.
* Easily override almost all behavior with custom code or
configuration variables
* And much more...
Once you're up and running, it's SQL Alchemy under the hood, with a
few tune-ups that you are free to bypass if you don't like them.
Home page: http://www.turnkey-analytics.com/struqtural/
PyPI: http://pypi.python.org/pypi/Struqtural
Source: https://code.launchpad.net/pydatastep
Bugs: https://bugs.launchpad.net/pydatastep
To see Struqtural in action, take a look at the tutorial...
=
Struqtural has been designed to be as smart as possible. Because of
that, most common use cases are retardedly simple. Let’s take a look,
shall we?
First, let’s just examine the most basic use case:
from struqtural.structures.structure import Structure
from struqtural.loaders import loader
data = [{A:True, B:1, C:1.5, D:a},
... {A:False, B:2, C:2.5, D:b},
... {A:True, B:3, C:3.5, D:c},
... {A:False, B:4, C:4.5, D:d}]
example_structure = Structure(loader.CollectionLoader, data, B,
SimpleInstance)
print primary keys:, example_structure.primary_keys
primary keys: ['B']
# Note that you could have also specified the primary key using an iterable
print table name:, example_structure.table
table name: SimpleInstances
print table columns:, example_structure.columns
table columns: ['SimpleInstances.A', 'SimpleInstances.C',
'SimpleInstances.B', 'SimpleInstances.D']
for true_instance in example_structure.filter(B2):
... print true_instance
SimpleInstance(A=True, C=3.5, B=3, D=c)
SimpleInstance(A=False, C=4.5, B=4, D=d)
for small_true_instance in example_structure.query().filter(A, C3):
... print small_true_instance
SimpleInstance(A=True, C=1.5, B=1, D=a)
Another nice feature is that Struqtural is pretty flexible about how
you pass it data:
from struqtural.structures.structure import Structure
from struqtural.loaders import loader
more_data = [[T, 1, 1.0],
... [FALSE, 2, 2.0],
... [true, 3, 3.0],
... [False, 4, 4.0]]
# Note that more_data is strings, which isn't uncommon in the real world
data_headers = [A, B, E]
categories = Structure(loader.CollectionLoader, (data_headers, more_data),
[B, E], Category)
# Just a quick aside to demonstrate that Struqtural gets your table names right
print table name:, categories.table
table name: Categories
for category in categories:
... print category
...
Category(A=True, B=1, E=1.0)
Category(A=False, B=2, E=2.0)
Category(A=True, B=3, E=3.0)
Category(A=False, B=4, E=4.0)
As you can see the strings have been handled elegantly. Type
conversion is completely controllable and extensible. A small
collection of useful type inference and conversion functions have been
included.
How about if we want to create a new structure out of multiple
previously existing ones:
example_structure = Structure(loader.CollectionLoader, data, B,
SimpleInstance)
connector = example_structure.connector
categories = Structure(loader.CollectionLoader, (data_headers, more_data),
[B, E], Category, connector)
joint_structure = Structure(loader.JoinLoader, categories,
example_structure)
for instance in joint_structure:
... print instance
Category(A=True, B=1, E=1.0, C=1.5, D=a)
Category(A=False, B=2, E=2.0, C=2.5, D=b)
Category(A=True, B=3, E=3.0, C=3.5, D=c)
Category(A=False, B=4, E=4.0, C=4.5, D=d)
Note that we took advantage of some intelligence on the join we did
there, since we have identically named, identically typed columns in
the two tables. If you needed to be explicit about it you could just
as easily have called structure like so (where the dictionary key is
the column on the left table, in this case categories, and the value
is the column on the joined table):
joined_with = (example_structure, {'A':'A', 'B':'B'})
joint_structure = Structure(loader.JoinLoader, categories, joined_with)
Next, let’s create persisted versions of
Re: Why doesn't python's list append() method return the list itself?
The better question is, do I ever use them? Thinking back over the code I've written in the last couple of years, I would say probably two or three times (mostly in unit tests). I've had to code around string's sequence behavior DOZENS of times. Is it nifty that strings can be sliced like that? Sure. In general my experience has been that the string methods (and some misc functions in other modules) do a better job in simple cases, and regular expressions do a better job in complex cases. I just don't see string slicing having utility that is irreplaceable, and it is bug-inducing in many cases, or at the very least it can propagate programming errors way down stream. Ultimately, I highly doubt it's going anywhere, so it's a moot point. I definitely feel that python is trading real usability for flash with string slicing though. On Tue, Jul 13, 2010 at 8:48 PM, Aahz a...@pythoncraft.com wrote: [Original not available on my swerver, responding here] On 7/11/10 10:03 PM, Nathan Rice wrote: Yeah, I long ago filed the in place place in the same folder as strings-as-sequences, all() returning True for an empty iterable and any returning True rather than the thing which triggered it. Because I love to repeat myself: ...string iteration isn't about treating strings as sequences of strings, it's about treating strings as sequences of characters. The fact that characters are also strings is the reason we have problems, but characters are strings for other good reasons. --Aahz http://mail.python.org/pipermail/python-3000/2006-April/000897.html Do you really want to give up Python's lovely string-slicing capabilities? -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ Normal is what cuts off your sixth finger and your tail... --Siobhan -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't python's list append() method return the list itself?
Stephen: I'm not adverse to being able to do that, but the number of times that I've wanted to do that is greatly outweighed by the number of times I've had to pass a function (somestring,) or call if isinstance(foo, basestring): ... to avoid producing a bug. The more abstract and adaptive the code you are writing, the more annoying it gets - you end up with a rats nest of string instance checks and strings wrapped in tuples. Looking at my code, I don't have a lot of use cases for string slicing or iterating character by character. Most of the time I use the string methods, they're faster and (IMO) clearer - lower, index/rindex, find, etc. One use case that I avoid is extracting substrings, by slicing out the results of rfind. There's a good case for this but I feel it's brittle so I usually just jump to regular expressions (and it could be performed equally well with a substring method). That doesn't mean I don't think it's useful, just that as it stands the default language behavior is bug producing, and in my opinion people would be equally well served with an as_list method on strings that makes the behavior explicit. Chris: Let's not run around questioning people's math skills, that's actually my area of expertise, and it's impolite besides :) While having all([]) return True from a formal standpoint makes sense it typically reduces people to writing if all(something) and something:, because feeding an iterable that has been filtered in some way (and thus has a range between 0 and n, where n is the length of the original iterable) is an incredibly common use case. In fact, I'm going to go out on a limb here and say there are a lot of bugs floating around that haven't been caught because the code author used all() under the assumption that it would be passed a non-empty list. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't python's list append() method return the list itself?
Do list(reversed(list(reversed([1, 2, 3, 4])) + [[]])) Though TBH sometimes get annoyed at this behavior myself. There are a lot of people who are very vocal in support of returning none, and it makes sense in some ways. Since reversed returns an iterator though, it makes this code horrible and unreadable. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't python's list append() method return the list itself?
Yeah, I long ago filed the in place place in the same folder as strings-as-sequences, all() returning True for an empty iterable and any returning True rather than the thing which triggered it. Almost always annoying and worked around, but that's the price you pay for the other nice stuff :) It just takes writing a few hundred lines of Java code for me to shrug and forget about it. -- http://mail.python.org/mailman/listinfo/python-list
Re: Opinions please -- how big should a single module grow?
I start to look at whether some subset of functions or classes are not referenced by other subsets of functions or classes in a module when it gets to about 1K LoC, and if I don't find any by the time it gets to about 1500 LoC, I start to look at ways I can refactor the code in the module to be less coupled. This might just be anecdotal, but in all the python libraries I've looked at (including the larger 30K+ LoC ones) the quality of code in a module tends to be lower if the module is over around 1500 LoC. -- http://mail.python.org/mailman/listinfo/python-list
Re: python app development
Expert Python Programming by Tarek Ziade is a fairly good book, covers a lot of core stuff, though it doesn't really cover gui app development at all. On Sat, Jul 3, 2010 at 1:48 PM, mo reina urban.yoga.journ...@gmail.comwrote: an anyone recommend a resource (book,tutorial,etc.) that focuses on application development in python? something similar to Practical Django Projects, but for stand alone applications instead of web apps (for now). i'm in a bit of a funny place, i have a decent/good grasp of python syntax and my logic isn't bad, but i have no clue on how to assemble an application, i seem to be stuck on writing scripts. i've looked at the source of a few projects but the flow is way over my head, i understand the syntax but not the logic, which is why i'm looking for a project-cenetered learning resource, instead of a reference or language-feature resource. also, it seems that a lot of app programming is 90% gui bindings, with very little actual code, or am i totally way off mark? i recently picked up the django practical projects book, and in a few days i re-wrote a website i did with django. i feel it was the book's project-centric approach that made this possible. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Decorators, with optional arguments
I like to think of decorators with arguments as decorator factory functions.
I try and unroll them as much as possible... I have some decorators that
work like so (and please note that the wraps and returns_as_output are
separate so that I can mutate the behavior as needed, if you just wanted a
single decorator this could be condensed):
def wrap(f, wrapping_function, *args):
argspec = inspect.getargspec(f)
without_defaults = len(argspec.args) - len(argspec.defaults or [])
f._args = {}
f._varkw = []
f._arg_defaults = (None,) * without_defaults + (argspec.defaults or
tuple())
for arg in args:
if arg in argspec.args:
f._args[arg] = argspec.args.index(arg)
else:
if argspec.keywords:
f._varkw.append(arg)
return decorator.decorator(wrapping_function, f)
def return_as_output(output_f):
def func(f, *args, **kwargs):
new_args = list(args)
for (dict_name, index) in f._args.items():
if len(args) index:
new_args[index] = output_f(args[index])
else:
d = f._arg_defaults[index]
kwargs[dict_name] = output_f(kwargs.pop(dict_name, d))
for dict_name in f._varkw:
kwargs[dict_name] = output_f(kwargs.pop(dict_name, None))
return f(*new_args, **kwargs)
return func
def iterables(*args):
'''
Decorator that guarantees that the named arguments will be iterable.
'''
as_iterables = return_as_output(iter_)
return lambda f: wrap(f, as_iterables, *args)
Just as an example... I have a family of decorators that check compliance
with abstract base classes, then try to mutate variables to provide it if
duck typing would fail. I don't know if this helps, it is kind of
convoluted, but hopefully it gives you another example to work from.
--
http://mail.python.org/mailman/listinfo/python-list
