[Python-Dev] xml.sax and xml.dom fetch DTDs by default (was XML DoS vulnerabilities and exploits in Python)
Perhaps related to the discussion of denial-of-service vulnerabilities is the matter of controlling access to remote resources. I suppose that after the following bug was closed, no improvements were made to the standard library: http://bugs.python.org/issue2124 Do Python programs still visit the W3C site millions of times every day to download DTDs that they are not, by default, able to remember from their last visit? Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new webpage promoting Compiler technology for CPython
Travis Oliphant wrote: With Numba and Blaze we have been doing a lot of work on what essentially is compiler technology and realizing more and more that we are treading on ground that has been plowed before with many other projects. So, we wanted to create a web-site and perhaps even a mailing list or forum where people could coordinate and communicate about compiler projects, compiler tools, and ways to share efforts and ideas. The website is: http://compilers.pydata.org/ This is a rather nice resource. Thank you for letting us know about it! There has been an attempt to record different Python implementations on the Python Wiki, and now that this is available again, I'd like to remind people about it: http://wiki.python.org/moin/PythonImplementations I know that this isn't quite the same thing as a page about compiler technology, but there is a substantial overlap. This page is specifically for Compiler projects that either integrate with or work directly with the CPython run-time which is why PyPy is not presently listed. The PyPy project is a great project but we just felt that we wanted to explicitly create a collection of links to compilation projects that are accessible from CPython which are likely less well known. I think that given the scope of the projects listed, it shouldn't preclude PyPy from being listed, really. After all, interoperability with CPython extensions is something of a focus area for PyPy: http://pypy.org/compat.html I don't have an agenda here - I don't use PyPy actively, my only involvement with Shedskin (which is referenced and which can produce CPython extension modules) is in packaging it for Debian, and although I do have a static analysis library I see no pressing need to promote it extensively - but I feel that when it comes to educational resources people should be a bit careful about drawing boundaries that exclude things that would benefit people substantially if they only knew about it. My reason for speaking up about this is that I've had to tell a room full of people who were told to use Cython, NumPy and even plain C to make their Python programs faster that PyPy existed. (Of course, one can justify ignoring the elephant in the room by claiming things like scientific users rely on native libraries or CPython extensions - since science is a very broad term, this obviously isn't universally true - but I think people should be entitled to make their own minds up, and I was not completely certain that everyone involved in the case in question was oblivious to PyPy's existence or status.) But that is just where we started from. The website is intended to be a community website constructed from a github repository. So, we welcome pull requests from anyone who would like to see the website updated to reflect their related project.Jon Riehl (Mython, PyFront, ROFL, and many other interesting projects) and Stephen Diehl (Blaze) and I will be moderating the pull requests to begin with. But, we welcome others with similar interests to participate in that effort of moderation. The github repository is here: https://github.com/pydata/compilers-webpage This is intended to be a community website for information spreading, and so we welcome any and all contributions. There is also the python-static-type-checking Google Group: https://groups.google.com/forum/?fromgroups#!forum/python-static-type-checking If no-one beats me to it, I may post details of the site to that group because it may well be of interest to the members. Thanks once again for bringing such information together in one place! Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new webpage promoting Compiler technology for CPython
Stefan Behnel wrote: This is off-topic for this list, but the main problem with PyPy is that you'll quickly hit a lot of walls when you try to use it for anything serious in the area. It's true that there is a certain level of interoperability with CPython extensions, but calling it a focus area makes it sound bigger than it actually is in my ears. It is a focus area. They even have a Web page for it, which I mentioned. Even trying to get bugs fixed to at least make things work at all often means running against walls on their side. I can tell, trying to port Cython mostly consisted of bugging PyPy developers to fix stuff, which took anything from days to still-not-done, each time. And, by design, PyPy makes it very hard and time consuming to debug it and to try to fix bugs in their code base. I'm sure it is tricky to debug PyPy. However, complete compatibility with CPython for Python programs is a goal of the project, and I have no reason to doubt that the project is committed to supporting broader compatibility with CPython. So, while I agree that PyPy is worth a try in certain application areas, and can be helpful for some special needs, also in the field of scientific computing, it's lightyears away from a production-ready state in that area. You are generalising too broadly, which is precisely what I mentioned in my last message. There are also plenty of people doing science who aren't using numeric libraries or relying on native code libraries. What I most took exception to was either the lack of awareness of PyPy amongst people giving advice on how to speed up Python programs - not specifically numeric programs - to an audience of people who happened to be scientists, or the pretense that the first port of call was to break out Cython or NumPy when the easiest thing for them to do was to try their code on PyPy, provided they could get a package for it. One of my colleagues thought that you could always rewrite your code in C, which is what he took away from such recommendations, was hilarious advice for speeding up Python programs. You write your Python code in another language! Why not just use C in the first place? Not such a stupid question, really. It's a question that has been hanging around for far too long without a really good answer. It just doesn't integrate with the huge bulk of software that people use in their daily work. And once you rely on that software, which is hand tuned, well integrated and real-world proven in so many ways, over the time span of way more than a decade, the most visible advantage of PyPy to make Python code run faster becomes almost pointless. In that light, telling people to try PyPy and to drop (most of) their current ecosystem for it doesn't really sound helpful and clearly appears outside of the focus of the web site in question. For a very long time, and even now to some extent, you could say the same thing about Python 3. Personally, I suspect that some people have such a problem with PyPy that they would rather not mention it or see it mentioned. That's obviously a shame because not only does PyPy offer people an alternative, but it also encourages the development of software in Python, some of which appears on the resource being discussed, that would otherwise be written in C because that's what we would usually write this kind of software in. And although I don't actively use PyPy, mostly because of what the default Python implementation is on the platforms I use, I would like to see more actual Python code written. But I'm certainly not going to dwell on this matter any more than I've already done. I'm sure people will get something from the referenced resource regardless of whether any particular project is mentioned by it or not. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FYI - wiki.python.org compromised
Robert Whitney wrote: To Whoever this may concern, I believe the exploit in use on the Python Wiki could have been the following remote arbitrary code execution exploit that myself and some fellow researchers have been working with over the past few days. I'm not sure if this has quite been reported to the Moin development team, however this exploit would be triggered via a URL much like the following: http://wiki.python.org/WikiSandBox?action=moinexecc=uname%20-a Did you check the MoinMoin security fixes page? http://moinmo.in/SecurityFixes What you describe is mentioned as remote code execution vulnerability in twikidraw/anywikidraw action CVE-2012-6081. This URL of course would cause for the page to output the contents of the command uname -a. I think this is definitely worth your researchers looking into, and please be sure to credit myself (Robert 'xnite' Whitney; http://xnite.org) for finding reporting this vulnerability. Have you discovered anything beyond the findings of the referenced, reported vulnerability, or any of those mentioned in the Debian advisory? http://www.debian.org/security/2012/dsa-2593 If so, I'm sure that the MoinMoin developers would be interested in working with you to responsibly mitigate the impact of any deployed, vulnerable code. Paul P.S. Although I don't speak for the MoinMoin developers in any way, please be advised that any replies to me may be shared with those developers and indeed any other parties I choose. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using Wiki for Python development coordination
On Saturday 03 November 2012 12:29:57 anatoly techtonik wrote: I don't have access to modify the front page. http://wiki.python.org/moin/FrontPage Yes, access was restricted a while ago because of spamming. To me it lacks one more section concerning help with core development infrastructure. Python Core Development Python Website http://wiki.python.org/moin/PythonWebsite The entrypoint for everything concerning the language Bug Tracker http://wiki.python.org/moin/TrackerDevelopment Roundup and code review services we use Package Index http://wiki.python.org/moin/CheeseShopDev Some building blocks for your own projects (including frameworks for database, GUI, Web programming) I'll admit that the current content is just a reformatted version of what was there before, but tidied up and making better use of vertical space, and it could be improved. Certainly, there isn't a core development section, so perhaps I'll just add what you suggest. I know how you dislike the Moin markup, Anatoly. ;-) Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bloody FAQ (Was: [Python-ideas] itertools.chunks(iterable, size, fill=None))
Stefan Behnel wrote: anatoly techtonik, 05.07.2012 21:41: Make the bloody FAQ and summarize this stuff? Why waste each others time? Yes, that is exactly the question. It takes time to write things up nicely. I mean, once someone has pointed out to you that this has been discussed before, you could just go, look it up (or search for it), and then put it into a Wiki or blog post yourself, or sum it up and send it to the mailing list as a reply. Why rely on others to do it for you? To be fair, Anatoly has done quite a bit of maintenance on some of the Wiki content around various aspects of the project, so it's not as if he's demanding anything out of the ordinary or asking for others to do things that he isn't already doing in some sense. My experience is that there usually needs to be some willingness on the other end of the transaction, and if it takes repetition to encourage it amongst those who don't see the current situation as a problem for them, then so be it. Of course, this kind of documentation activity, where one gathers together historical decisions and the consensus from long-forgotten discussions, is pretty thankless work. I occasionally regard it as worthwhile if only to bring up something someone said as an inconvenient interruption in any current discussion, but that's a pretty minimal reward for all the effort unless one has such work as part of one's daily routine. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Announcing the python-static-type-checking google group
Stefan Behnel wrote: Paul Boddie, 01.07.2012 02:22: Is there any reason why the compiler-sig mailing list wasn't chosen as a venue Even I didn't know that this list even existed. And looking at the archives now, it's hard to see any relevant discussion in all the spam it received until it apparently died away in (almost) silence a couple of years ago. Yes, although the mailing lists for special interest groups are advertised on python.org, there is no longer the focus on steering discussion to those lists. And I see that the compiler SIG is retired, as is the related types SIG: http://www.python.org/community/sigs/retired/ I seem to remember various procedures about SIGs and their retirement, but I don't really recall much discussion of such things recently. Still, the compiler SIG matches the scope of the Google group pretty well: http://www.python.org/community/sigs/retired/compiler-sig/ There's even a link to discussion of some tools you may be familiar with. It's obviously your choice where you host discussions and who you invite, and I know that the special interest group mailing lists aren't exactly well advertised these days True, but many (most?) of them are simply not very well frequented, which reduces the interest in joining them even further. Both SIG mailing lists that I read only receive a mail every so many months, often enough without any reply. And almost all of these mails deal with questions that would better be discussed on python-list to leverage the substantially higher number of eyeballs there. Special interest group lists were always meant to be used as focused channels of communication where people are actively trying to get stuff done. The unfortunate thing is that they aren't as well known as they were. Another unfortunate thing is that getting stuff done of mutual benefit is frequently something that takes second place to whatever other motivations and goals people have, for whatever reason, good or bad. Thus, traffic drops away as people either do other things entirely or instead promote any related work in other channels instead. I think that's the basic problem: as long as more experts are lurking on python-list than on the dedicated SIG-ML, it's better not to use the SIG-ML for discussions but to go to python-list (or maybe python-ideas or python-dev) straight away. I think we really have to sort out what python-dev is for, because currently there's a tendency to target the list when any kind of expert discussion is required, but there are a number of people who would rather see only CPython-related discussion here. Another matter is that static analysis of Python is a topic that frequently hits the end of the road when one cannot, by definition, analyze Python in its most dynamic form, and when people refuse to accept that such analysis has anything to do with Python in its most pure, undiluted (and most contrived) form. But as I wrote, I still intend to follow the newly created group and see what people have to say. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Announcing the python-static-type-checking google group
Edward K. Ream wrote: Hello all, GvR has asked me to announce the python-static-type-checking google group http://groups.google.com/group/python-static-type-checking to python-dev. Consider it announced. Anyone from python-dev who likes may become a member. Is there any reason why the compiler-sig mailing list wasn't chosen as a venue for such discussions? I know it has compiler in the title, but the mandate overlaps significantly with what you intend to discuss. It's obviously your choice where you host discussions and who you invite, and I know that the special interest group mailing lists aren't exactly well advertised these days, what with the lack of agility around updating the Web content that advertises such things (or the lack of visibility of the Wiki content), but I feel that you might have a more productive discussion if you don't insist on Google Groups membership and also allow a wider selection of participants. Please consider this as friendly advice: I too would like to see progress in the area concerned. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] I am now lost - committed, pulled, merged, what is collapse?
Skip wrote: Antoine wrote: 94 changesets? If you want to avoid risking conflicts, you should hg pull and hg up (or hg pull -u) before you start working on something (just like you svn up'ed before working on something). Sorry, this workflow is still new and hugely confusing to me. To make a simple edit to csv.rst I needed to do a couple pulls and merges, local commits, resolve the same conflict multiple times, get rebuffed when I first pushed because there was a tab in the file, and ask a question here. If this is the typical route necessary to make even the simplest changes which would have been a single commit with svn, my already meager rate of contributions is likely to wither away to nothing. This is reminiscent of a message on the Mercurial mailing list recently, to which I responded with a question about people experiencing problems with Mercurial because they are used to file- or directory-specific operations in other systems, eliciting this reply: hg failed saying there were uncommitted changes (his source code changes in another part of the tree). He didn't want to commit those changes yet, they weren't finished. So he was stuck. To his mind, hg was being stupid because it was getting in his way, unnecessarily linking changes in the two sets of files together. The concept of a revision being a state of the whole repo was alien. For most people that concept came in with svn. http://www.selenic.com/pipermail/mercurial/2011-March/037373.html I've spent some time trying to tidy up and improve a document on the Mercurial Wiki about CVS-like working practices with Mercurial, and this might explain the differences in operation between CVS/Subversion and Mercurial, although it doesn't yet distinguish between the narrow file-specific commits you get in systems like CVS and Subversion, and the whole-project commits you get in systems like Mercurial: http://mercurial.selenic.com/wiki/CvsLikePractice (By the time you look at that page, I might have added something, though.) I'm certainly no expert with Mercurial, and I've only been using it as a personal tool since the MoinMoin guys introduced me to it back at EuroPython 2006, so even now the right way or best practice when it comes to propagating fixes between branches/clones is something I'm still figuring out, but having lurked on the recent python-dev threads and having read various queries on the Mercurial list over the past year or so, I get the impression that reaching for things like rebase and mq as the first choice to solve various workflow problems would get some blunt criticism on the Mercurial list. That said, I can't readily find any good guides to managing a multi-branch project, but there seem to be some interesting techniques documented for some of the situations people are likely to encounter. For example: http://mercurial.selenic.com/wiki/DaggyFixes describes pulling fixes selectively which I'll probably have to try on some personal project at some point http://hginit.com/05.html describes the presumably common way of propagating fixes from stable branches to development branches Certainly, the Python devguide is a nice piece of documentation, but I feel that there's an opportunity here to address some documentation issues that would also help others using and adopting Mercurial. For example, submitting clean changes to a project (that collapse thing) is a topic for a document that could usefully be written, containing some nice diagrams that explain the mechanism to newcomers, and it would surely benefit more than just the CPython development effort. Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python wiki
Hello, I've been following this thread all week at work, but I thought it might be time to respond to the different remarks in a single response, given that I am involved in editing and maintaining the Wiki on python.org, and I had a strong enough opinion about such things to give a talk about it at EuroPython that some of you attended: http://wiki.europython.eu/Talks/Web_Collaboration_And_Python/Talk Guido van Rossum wrote: On Thu, Sep 23, 2010 at 3:35 PM, Martin v. Löwis martin at v.loewis.de wrote: There actually is an admin team, and they actually do set ACLs. Who are they? IIUC, this is primarily for spam protection, though. So would they object against additions to the team? The administrators are generally the people on the following page: http://wiki.python.org/moin/AdminGroup Someone may have special powers, particularly if they have shell access to the machine running the Wiki, but there's no secret brotherhood. In fact, I recall advocating that Carl Trachte get admin powers given his continuing high-quality contributions, so we can say that people aren't denying others administrative privileges if that person is clearly doing good work and would benefit from being able to have those privileges. [Later on...] Guido van Rossum wrote: On Fri, Sep 24, 2010 at 3:31 AM, Michael Foord fuzzyman at voidspace.org.uk wrote: Wiki maintenance is discussed, along with other python.org maintenance topics, on the pydotorg-www mailing list: http://mail.python.org/mailman/listinfo/pydotorg-www Which has hidden its membership (even to members). Does it really need to appear that secretive? At least the message archive is open and has all the usual suspects. As we're now seeing, people don't feel that it's acceptable to publish the subscribers list, and I think that it's a complete distraction to seek to do so, anyway. It's not as if everyone on that list has special privileges and is preventing newcomers from joining it. More wiki and website maintainers needed! Maybe a prominent wiki page with info about how to join the list and the responsibilities / needs would help? Also, I gotta say that the wiki login process is awkward. MoinMoin supports OpenID, although I did find and report issues with Moin 1.9 in this regard. Something on my now-huge list of things to do is to make Moin authentication better, especially where there might be a choice of authentication methods. Georg Brandl wrote: Am 23.09.2010 22:25, schrieb anatoly techtonik: On Thu, Sep 23, 2010 at 6:57 PM, Barry Warsaw barry at python.org wrote: I certainly agree with that. So, how can we solve those problems? Radomir has shell access now so perhaps we can ask him to make the Python wiki theme more visually appealing. What roadblocks do people encounter when they want to help garden or reorganize the wiki? First of all Wiki is outdated. Correct me, but python.org specific customizations are not modules, but patches that makes it hard to update the Wiki to latest version or add new customizations. That's why we have a Moin core developer on the team. ISTM that Moin 1.x is notoriously hard to extend -- once you go beyond plugins adding new wiki macros -- which is part of what the team wants to fix in 2.x. Although I understand the sentiments about specific customizations, python.org should only have its theme as something that isn't a generic extension to MoinMoin, and that theme should be actively maintained. When python.org switched its architecture a while ago, the special theme presumably came with the deal, but it's been so out of date for so long that I switched to one of the standard themes years ago. Fortunately, Radomir's EuroPython theme is actively maintained, works with recent MoinMoin releases, looks a lot better than the old theme, and is used elsewhere. As for Moin 1.x being notoriously hard to extend beyond macros, you can get a long way with macros and actions, although I agree that sometimes it's possible to hit architectural constraints. Second - ugly Creole syntax. I am for inter-cooperation between wikis, and I understand that for non-developer communities [] symbols imposing problems, but as an open source developer I am addicted to Trac and Google Code syntax and never had problems with those. This isn't Creole syntax, it's Moin wiki syntax. And face it, it's not going to change. It's also not so much different from Trac wiki syntax. I never agreed with this complaint, anyway. When discussing this previously with Anatoly, I went as far as to ask on #moin where Radomir actually posted a good summary of the syntax differences: http://wiki.python.org/moin/SiteImprovements/WikiSyntaxComparison I invite anyone to justify claims that the old style (which Trac adopted) was better. Complaints about double bracketing are specious: MediaWiki has different bracketing levels and it's really confusing. Fourth. GPL license.
Re: [Python-Dev] as keyword woes
On Monday 08 December 2008 22:54:41 Guido van Rossum wrote:
From my experience with SQL, it's nearly as bad as Python in that
every single one of the 200+ reserved words in a typical
implementation cannot be used as a name in any context without using
double quotes.
SQL is a big language; I won't disagree with that! That said, you don't always
have to quote names like end as I mention below.
While the double-quote escape is handy (especially
given there are so many obscure reserved words) this is not exactly
what the OP wanted -- they would have to say x.as('float'), except
using some other notation instead of double quotes. Having to escape
it completely kills the OP's claim that 'as' is simplest and most
elegant.
You can do what the OP wants, at least in PostgreSQL, which is fairly
conformant. As I wrote on comp.lang.python...
create table create (
select varchar
);
select select from create;
select create.select from create;
(This from a PostgreSQL 8.2 session.)
I don't know whether SQL 1992 actually allows dropping the double-quotes for
column names, but this is the kind of thing he has in mind.
Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] as keyword woes
On Sat Dec 6 21:29:09 CET 2008, Guido van Rossum wrote: On Sat, Dec 6, 2008 at 11:38 AM, Warren DeLano warren at delsci.com wrote: As someone somewhat knowledgable of how parsers work, I do not understand why a method/attribute name object_name.as(...) must necessarily conflict with a standalone keyword as . It seems to me that it should be possible to unambiguously separate the two without ambiguity or undue complication of the parser. That's possible with sufficiently powerful parser technology, but that's not how the Python parser (and most parsers, in my experience) treat reserved words. Reserved words are reserved in all contexts, regardless of whether ambiguity could arise. Just a quick aside from someone who merely lurks on this list: in SQL, it's quite possible to use keywords in a fashion similar to that desired by the inquirer, and it's actually possible to double-quote keywords and use them as names for things. I'm not advocating more complicated parsing technology for any Python implementation, but I think it's pertinent to point out that the technology isn't particularly obscure. Apologies for the interruption, Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
