Re: [Twisted-Python] Twisted book(s)?
While its a bit out of date now, and it certainly could have been better, I
found the book useful to get me over the initial learning curve hump that
twisted is famous for.
And it IS the only Twisted book out there (afaik).
Kevin Horn
On Tue, Jun 2, 2009 at 9:57 PM, Sury Soni ss...@nextdigital.com wrote:
You are right David.
I read that book and that book is definitely not for the beginner python
programmers.
And, yes, that book is not up to date as well.
Cheers.
Surya
*From:* twisted-python-boun...@twistedmatrix.com [mailto:
twisted-python-boun...@twistedmatrix.com] *On Behalf Of *David Conradie
*Sent:* Wednesday, 3 June 2009 12:55 PM
*To:* twisted-python@twistedmatrix.com
*Subject:* [Twisted-Python] Twisted book(s)?
I'm fairly new to Python and very new to Twisted. I find I learn new
material best from a book, so although I know there's lots of great
documentation and examples on twistedmatrix.com, I was considering getting
O'Reilly's Twisted book ('Twisted Network Programming Essentials' by Abe
Fettig). But that book dates back to Oct 2005 so I was wondering if it's
still a worthwhile purchase - I'd guess Twisted has moved a good bit since
the book was written. Any opinions?
Thanks
*David Conradie*
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted book(s)?
Yeah, those reviews are excellent, but I couldn't remember where I'd read
them.
The O'Reilly book is ok if you want a lot of VERY basic examples of how to
do stuff in Twisted. I also found the introduction somewhat useful, even
though I knew most of it.
I haven't read the Foundations of Python Network Programming book, but I've
flipped through it a couple of ties at the bookstore.
I decided against it, because I basically knew all the stuff in the
non-twisted chapters, and I didn't want to buy the whole book for just the
Twisted stuff. Since you're new to Python as well as Twisted it would
probably be much more useful to you. I've heard lots of good things about
it.
Either way, though, you aren't going to get a whole lot more than an intro
to Twisted in either one, mostly because, well, Twisted is HUGE. It does a
vast number of things, lots of it you'll probably never have to deal
with...except when you do.
The online docs are pretty good as far as accuracy...most of it is pretty up
to date (or at least up to date ENOUGH), but it's organization is kind of
wacky. At least is doesn't fit my brain very well...I have trouble finding
things.
I would love to see another Twisted book, but it's kind of a small niche to
get publishers to go for. Maybe an online book, or something published
on-demand could work, though.
Kevin
On Wed, Jun 3, 2009 at 2:26 AM, Reza Lotun rlo...@gmail.com wrote:
You may also find Foundations of Python Network Programming useful -
it has large sections on twisted, particularly IMAP
(http://www.complete.org/publications/pynet/)
You can also read these useful reviews of the above as well as the
O'Reilly book, written by a Twisted dev:
http://eagain.net/articles/review-pynet/
http://eagain.net/articles/review-snakeball/
Cheers,
Reza
On Wed, Jun 3, 2009 at 6:12 AM, Robert Gravina rob...@gravina.com wrote:
I'm fairly new to Python and very new to Twisted. I find I learn new
material best from a book, so although I know there's lots of great
documentation and examples on twistedmatrix.com, I was considering
getting O'Reilly's Twisted book ('Twisted Network Programming
Essentials' by Abe Fettig). But that book dates back to Oct 2005 so I
was wondering if it's still a worthwhile purchase - I'd guess Twisted
has moved a good bit since the book was written. Any opinions?
I got that book and to be honest, didn't find it very useful due to it
being a nutshell book - it can't (due to size) go into much detail in
each section and so I found the online documentation/tutorials were a
better resource.
A more detailed Twisted book would be awesome though! I think
pragprog.com share 50% of the profits with authors (and do ebooks and
betas too), so maybe someone might be interested in writing it :)
Of course noone would complain about a community edited book like what
the Django project has, either. :)
Robert
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
--
Reza Lotun
+44 (0)7521 310 763
rlo...@gmail.com
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] AsyncQueue is still unavailable
I have no idea what happened to the site (archive.org shows no updates after Feb 2008), but older versions of both sAsync and AsynQueue are available on PyPI: http://pypi.python.org/pypi/sAsync/0.7 http://pypi.python.org/pypi/AsynQueue/0.3 They look to have been written for Py2.4, but perhaps someone could update them? Kevin Horn On Thu, Jun 18, 2009 at 5:07 AM, mardiros mardi...@laposte.net wrote: I everyone, I have too write a network application, and I think twisted is the best choice for it! I need to do some sql transaction, and I prefer too use an ORM too request the DB. But the server I have to connect is an SqlServer, so sqlalchemy is, I think the best choice. And every google search get back to AsyncQueue http://foss.eepatents.com/AsynQueue But the site is done since a few moment. Does anybody have news about this project ? Is it still alive ? Which alternative ? Thanks. mardiros *Créez votre adresse http://www.laposte.net électronique prenom@laposte.net 1 Go d'espace de stockage, anti-spam et anti-virus intégrés.* ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Need help with writing a secure Iran Proxy
Of course it depends on you definition of safe and secure, but there's a few examples around the net. Here's a few: http://wiki.python.org/moin/Twisted-Examples http://www.linuxjournal.com/article/7871 http://proxies.xhaus.com/ If you want to add proxy authentication, it shouldn't be too difficult...check out RFC 2068. http://www.ietf.org/rfc/rfc2068.txt (esp sections 10.4.8, 11, and 14.33-34) It should be similar to implementing basic HTTP auth, except you want to use the Proxy-Authenticate and Proxy-Authorization headers instead of the regular ones. I'm guessing the flow would go something like: - client requests a proxied resource ( without a Proxy-Authorization header ) - server sends back a 407 with a Proxy-Authenticate header - client prompts user for credentials - client provides request again with Proxy-Authorization header - server proxies the resource of course I'm no expert on web proxies so don't take my word for it... I think you should be able to do the above by overriding some of the proxy classes in twisted.web.proxy Good Luck! Kevin Horn On Thu, Jun 18, 2009 at 2:44 AM, Andreas Trawoeger atra...@gmail.comwrote: Hi! Could I kindly ask all the Twisted Gurus to take a look Austin Heaps Best Proxy Practices for Iran http://blog.austinheap.com/ The current setup (Squid iptables) everybody uses is extremely complicated to setup and it should be trivial to implement the requirements in Twisted. Turn it into an egg and allow everybody to simple type easy_install iranproxy. I know a little bit of Twisted, because I'm an avid Zenoss user and in different circumstanced I would write the proxy on my own. But in this case a safe implementation is a real must and I'm really worried to make some stupid beginner mistakes. So some (example) code and links to websites I should read would be greatly appreciated. Thanks to everybody! cu andreas ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] PLAINAuthenticator in twisted.mail.imap4
I was digging through the Twisted IMAP code tonight and I noticed something puzzling... PLAINAuthenticator.challengeResponse() uses the following statement to send auth credentials to the server return '%s\0%s\0' % (self.user, secret) which would give auth credentials of the form: authidNULpasswordNUL (where NUL is the NUL character) However, both RFC2595 and RFC4616 (both define the PLAIN SASL mechanism), say that credentials should be passed this way: [authzid]NULauthnidNULpassword (where NUL is the NUL character and [authzid] is optional) Now even if one was to leave the authzid out of the equation, you would end up with something like this: NULauthnidNULpassword and the version Twisted's IMAP code uses appears to be invalid. Am I crazy? Am I missing something? Is it just way too late and I should put the RFCs down and back away slowly? Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] PLAINAuthenticator in twisted.mail.imap4
On Wed, Jul 29, 2009 at 11:03 AM, Kevin Horn kevin.h...@gmail.com wrote: On Wed, Jul 29, 2009 at 10:51 AM, Kevin Horn kevin.h...@gmail.com wrote: On Wed, Jul 29, 2009 at 6:29 AM, Jean-Paul Calderone exar...@divmod.comwrote: On Wed, 29 Jul 2009 00:54:20 -0500, Kevin Horn kevin.h...@gmail.com wrote: I was digging through the Twisted IMAP code tonight and I noticed something puzzling... PLAINAuthenticator.challengeResponse() uses the following statement to send auth credentials to the server return '%s\0%s\0' % (self.user, secret) which would give auth credentials of the form: authidNULpasswordNUL (where NUL is the NUL character) However, both RFC2595 and RFC4616 (both define the PLAIN SASL mechanism), say that credentials should be passed this way: [authzid]NULauthnidNULpassword (where NUL is the NUL character and [authzid] is optional) Now even if one was to leave the authzid out of the equation, you would end up with something like this: NULauthnidNULpassword and the version Twisted's IMAP code uses appears to be invalid. Am I crazy? Am I missing something? Is it just way too late and I should put the RFCs down and back away slowly? My early morning reading of the RFC agrees with yours. Someone else brought this up a long time ago, I think, but never pointed out the RFC. Can you file a ticket? Jean-Paul At least I'm not crazy... :) Ticket #3939 filed: http://twistedmatrix.com/trac/ticket/3939 also added a note in the ticket that PLAINCredentials may need to be modified to match Kevin Horn FYI, attached a patch to the ticket. I haven't really tested it, but if someone could take a look and let me know what they think I'd appreciate it. Kevin Horn Can anyone tell me what the recommended way to run the twisted test suite against my trunk checkout is (on Win32)? I can't seem to make it work. I just get a bunch of DeprecationWarnings and then a stack trace complaining about not being able to remove my _trial_temp directory... Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] PLAINAuthenticator in twisted.mail.imap4
On Wed, Jul 29, 2009 at 11:55 AM, Kevin Horn kevin.h...@gmail.com wrote: Can anyone tell me what the recommended way to run the twisted test suite against my trunk checkout is (on Win32)? I can't seem to make it work. I just get a bunch of DeprecationWarnings and then a stack trace complaining about not being able to remove my _trial_temp directory... Kevin Horn Doh! Forgot to run 'setup.py develop'... OK, tests have been run, ticket updated. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Deferred documentation rewrite
This looks really great...looking forward to the results! Kevin Horn On Thu, Jul 30, 2009 at 2:03 PM, Jamu Kakar jka...@kakar.ca wrote: Hi Edward, On Thu, Jul 30, 2009 at 11:24 AM, Edward Z. Yangezy...@mit.edu wrote: I have been prodded by the members of #twisted into rewriting the Deferred documentation. You can check out the plan at this ticket: http://twistedmatrix.com/trac/ticket/3943 Comments would be appreciated. Your outline looks nice. Something that *really* helped me a lot with Deferreds was seeing how they are modelled after standard Python flow control behaviour. I guess that's what the first section your proposing is about. Jono Lange gave a presentation recently (can't remember what it was called... maybe something about being an evil hacker or about how your code sucks and he hates you) where he presented step-by-step slides that shows some normal Python code and then the asynchronous Deferred-using equivalent. Even though I understood the principles reasonably well before attending his talk, the way he presented them in his slides was very effective and helped me clarify that understanding. If he's willing, which I suspect he will be, I recommend you look at the slides and steal his good ideas. :) Thanks, J. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Missing Interface?
the following classes: twisted.cred.credentials.CramMD5Credentials twisted.mail.imap4.PLAINCredentials twisted.mail.imap4.LOGINCredentials all seem to implement the same interface, yet there is no Interface defined for these objects. But Kevin, you might be thinking, They all implement twisted.cred.credentials.IUsernameHashedPassword or t.c.c.IUsernamePassword. That's true, but they all have additional methods which make them useful for handling SASL authentication in various protocols. And those methods aren't included in those interfaces. So, should there be an Interface which describes the interface that these objects implement? Would that make it easier for protocol implementors to deal with SASL in their protocols? If it is desirable to have such an Interface, what should it be called? Would this be of any use to anyone out there? (besides possibly me) And why are two of these in the IMAP package and the third in cred? Why not in the same place? Thoughts? Anyone? Bueller? Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Questions about adding documentation
, but I found it pretty much incomprehensible. Thanks! Jean-Paul A couple of other things: There are several barriers to improving Twisted's docs - you have to know a fair amount about Twisted to write about some parts of it - you have to know a lot about Twisted to write about Twisted from a high level, big-picture sort of perspective - you have to know some seriously deep Twisted magic to even speak intelligibly about certain things - once you know enough about Twisted to really explain it...it seems you are no longer able to explain it :) Not trying to offend...everyone in the Twisted community I've talked to has been very helpful, but sometimes when you know a complex topic really well, it becomes difficult to explain things in such a way that someone new to that topic will understand. Does this make sense? Sometimes I think what is needed is someone who knows basically nothing about Twisted to go about learning it...and then writing down how they did it. Also: Some (more) documentation about how to write docs for Twisted would really help people who are willing (and in some cases eager!) to contribute, but feel the barrier to entry is too high (they don't want to figure out pydoctor + epydoc + xml processing + ...). Anyway, I hope my rambling is helpful... Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Questions about adding documentation
On Fri, Jul 31, 2009 at 9:34 AM, Phil Christensen p...@bubblehouse.orgwrote: On Jul 31, 2009, at 10:19 AM, Santiago Aguiar wrote: It could also be a good idea to try to make it more wiki-like at the beginning; I think there's quite a lot of people that would like to contribute with this, but having different documents/styles for different areas of the doc is not good, a wiki like doc might allow some people to work on giving a more cohesive look to the whole doc, and others to add the meat and bones, provide examples, etc, while allowing everyone to review it easily. Thinking about it, of course a doc on SVN would help in the same way ;). I'd like to second this, for at least the third or fourth time over the last several years ;-) To respond in advance to previous criticisms of the idea: * wiki syntax sucks * it doesn't have an acceptable form of version control * it's frustrating that it can't be easily bundled with a release tarball * the same info is on the mailing list, just use google to find it All 100% true. Nevertheless, I say wiki. It would provide the least complicated route for new users to document things they discover during the learning process, which is *exactly* the phase that is so hard for the more experienced users to properly explain. Hopefully over time the various technical writers who volunteer often could rewrite this content into the formal collection that is distributed with Twisted proper, but in the meantime, a community- edited wiki would also provide another window into the community for new developers. I also would recommend MediaWiki. Yes, less than perfect, not as good as this, that, or the other wiki. In the interest of keeping the bar for entry low, however, we would be harnessing a familiarity with MediaWiki provided by exposure to Wikipedia. Pragmatism, not idealism, and all that... -phil Well, if there *were* to be a change in the documentation system used, I would push for moving to Sphinx, rather than a wiki. * it's rapidly becoming a standard for docs in the Python world * it has lots of neat features * it can be version controlled * multiple output formats (html, chm, Latex(and therefore PDF), etc.) * I just like it :) Just sayin' Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Deferred documentation rewrite
On Fri, Jul 31, 2009 at 9:07 AM, Terry Jones te...@jon.es wrote: Michael == Michael Hudson mica...@gmail.com writes: Michael The accompanying paper is here: http://mumak.net/stuff/twisted-intro.html I agree, that's a really nicely done document. It would be good to extend it, moving into a description and explanation of deferreds. Terry Yes, indeed, a very nice paper. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Questions about adding documentation
On Fri, Jul 31, 2009 at 11:14 AM, Santiago Aguiar santiago.agu...@gmail.com wrote: Reza Lotun wrote: Rather than change the documentation system entirely, who don't we just create a new resource for Twisted documentation - a Twisted Documentation Wiki. At the very least it can be a repository of various tips and tricks we happen to find, and maybe it can even be a staging ground for documentation that makes it into Twisted proper. Even if I re-suggested the wiki based documentation, I think it's important to be extra careful on how it's used. One thing I personally hate is projects whose documentation is basically wiki-based, and what you end having is a disconnected set of tips, many out of date, of how to do this and that. It could be OK it it's labeled 'Tipi-wiki' but not 'Documentation' :). In my experience, this what happens to every project that a) uses a wiki for core documentation and b) has more than one person editing the wiki. As you say, I think that the wiki could be useful as a staging ground to build a formal documentation, but it shouldn't be even suggested for new users, since what they will probably find are many unfinished ideas. Alternatively, a separate doc repo with sphinx based doc could be built so that it will allow for collaborative development making clear that is a work in progress and a product with 'releases'. I say a different repo to avoid having to give commit access to code for people who are working on doc, maybe same repo with different permissions would be better. And I suggest sphinx to a) start from scratch and add existing doc as we see it's relevant, b) have a more flexible base than HTML docs, and for the reasons mentioned by Kevin (plus I want to learn it :P). -- Santiago. I'd love to see a documentation reboot using Sphinx, but not if it's going to be a half-baked, never-finished project. Since Jessica started this thread, and is the only person (so far) who has given any sort of commitment to actually working on new/revised docs, I'd really like to hear her opinion. Jessica? You still out there? Hope we haven't scared you off... I'd also be interested in hearing the opinions of some of the core Twisted guys on the various things we've been talking about here. What do you guys think about using a different docs system than what is being used now? If you guys are all dead set against it, there's not much point hashing out the details... Also, what do the Twisted core devs think about having a secondary wiki/cookbook thingy outside of the core docs? Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Questions about adding documentation
On Fri, Jul 31, 2009 at 11:52 AM, Phil Christensen p...@bubblehouse.orgwrote: My only question about Sphinx, isn't it just for API docs? Also, can it interpret Zope interfaces like pydoctor can? Sphinx is great for writing all sorts of docs. At its core, it's basically a system for gluing together a bunch of RestructiredText documents and automatically cross-linking them, building an index, a javascript-based search capability, handling tables of contents, and outputing the results in various formats. But then it's extensible...so you can write plugins for it. One of the earliest (and included with the base package) is the autodoc extension which makes generating nice clean API docs from your docstrings. What it does _not_ do is process all those epydoc bits that are in Twisted's docstrings, though you might be able to write an extension for that. It expects docstrings to be in RestructuredText. AFAIK Sphinx does not have support for Zope interfaces, but I haven't looked in a while, and I seem to remember that someone was working on adding it...though I may be making that up. At any rate, I think using Sphinx for Twisted's API docs (at this juncture, anyway) would be a bad idea. But Sphinx excels at the kind of long-form, instructional docs you mention below. I use it for all my PHP projects (which of course, the autodoc extension doesn't work on), and I'm really happy with it. Personally I'm pretty happy with the API docs (although there's always room for improvement in the actual docstrings), I think if there's a documentation need that's more dire, it's the long-form instructional kind. Agreed. I just don't want to sidetrack *that* discussion by getting into API documentation concerns. Double Agreed. -phil Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Questions about adding documentation
On Fri, Jul 31, 2009 at 12:11 PM, Jean-Paul Calderone exar...@divmod.comwrote: On Fri, 31 Jul 2009 11:55:08 -0500, Kevin Horn kevin.h...@gmail.com wrote: [snip] I'd love to see a documentation reboot using Sphinx, but not if it's going to be a half-baked, never-finished project. [snip] I'd also be interested in hearing the opinions of some of the core Twisted guys on the various things we've been talking about here. What do you guys think about using a different docs system than what is being used now? If you guys are all dead set against it, there's not much point hashing out the details... I don't find that people trying to use Twisted complain about the presentation of the documentation. I find that they complain about its content. True enough. And in my case, at least, navigation. So, I think that it's the content that needs to be addressed. I don't *think* that switching to Sphinx (or anything else) is going to make any different to the content of the current documentation. One might argue that Lore is a significant barrier to entry for new contributions to the documentation, but you'd have to try pretty hard to convince me. Pretty much anyone can write some simple (Lore x)html. And if they can't, then there are other people willing to translate plain text into Lore input documents. I don't think using Lore is a problem particularly...but it need to be documented more clearly, and all in one place. Right now, there's stuff on the tracwiki, in the main docs, on the mailing list, etc. Particularly, the answers to the questions: - How do I build the documentation using Lore? - What is the Lore xhtml syntax, and how should I use it? - What is the process for making a documentation contribution? Please note, that I was originally suggesting Sphinx primarily as an alternative to a wiki-based system, rather than as a replacement for Lore...but the conversation kind of mutated. :) Also, what do the Twisted core devs think about having a secondary wiki/cookbook thingy outside of the core docs? As a staging area for development of future core docs, I think I would recommend using a version control system (perhaps a distributed one), not a wiki. Agreed, wiki = yuck (for this). Even as a staging ground. IMO trac-wiki is really only suited for marketing type content. As something intended to be user-facing, I don't think it's a great idea. Of course, there *is* a wiki hosted on the website already... And it has some documentation on it... So what's being proposed, exactly? :) Obviously I'm speaking for myself, not for all the other people who have contributed to Twisted. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Questions about adding documentation
On Fri, Jul 31, 2009 at 12:11 PM, Kevin Horn kevin.h...@gmail.com wrote: On Fri, Jul 31, 2009 at 11:52 AM, Phil Christensen p...@bubblehouse.orgwrote: My only question about Sphinx, isn't it just for API docs? Also, can it interpret Zope interfaces like pydoctor can? AFAIK Sphinx does not have support for Zope interfaces, but I haven't looked in a while, and I seem to remember that someone was working on adding it...though I may be making that up. OK, having Googled around, it seems I was *not* making this up. There is some support for Zope interfaces in Sphinx, though it looks pretty minimal at the moment. Just, you know, in case anyone was interested. :) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Questions about adding documentation
On Fri, Jul 31, 2009 at 12:57 PM, Phil Christensen p...@bubblehouse.orgwrote: On Jul 31, 2009, at 1:34 PM, Kevin Horn wrote: Also, what do the Twisted core devs think about having a secondary wiki/cookbook thingy outside of the core docs? As a staging area for development of future core docs, I think I would recommend using a version control system (perhaps a distributed one), not a wiki. Agreed, wiki = yuck (for this). Even as a staging ground. Okay, tell me again what exactly the problem is with a wiki? I feel like the same thing happens every time we discuss documentation. Someone makes a recommendation to do it the easy way, and someone else dismisses any solution that doesn't satisfy their programmer's OCD. Yes, I'd love to see version controlled XML documentation that adheres to a common writing style that is enforced across the board, but this talk has been happening for *YEARS* and there has been little improvement of significance (I have to emphasize that I understand a number of people have worked very hard on this, and I don't mean to denigrate their contributions). People keep telling me wikis are bad, but I'm still not getting the 'why' -- I just hear wikis are bad for documentation repeatedly presented as a fact. They do seem to work reasonably well for scores of other projects. PRAGMATISM!!! ;-) -phil Well, pragmatism is my basic reason for not liking wikis for docs. :) For me, it boils down to every time I've worked on or with a project that used wikis for docs (assuming that the project is of at least moderate size and has more than a couple of editors) that documentation has been terrible. It's ended up as a disjointed mess, that's hard to navigate, and has a bajillion half-completed thoughts, and lots of outdated cruft. Of course, some could say the same of (at least parts of) the current Twisted documentation. ;) It's not that there's anything wrong with wiki's per se, it's just that they encourage bad habits. If there were a solid editorial process in place, where someone was specifically responsible for reviewing, editing, splitting, merging, and correcting documents, then a wiki could probably work. But I think it's easier to get coherent docs using tools that encourage good habits. Obviously my definitions of good habits and bad habits are pretty vague...and not entirely spelled out even in my own mind. Er, sorry about that... Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Really Basic clarification on defers
On Tue, Aug 4, 2009 at 10:08 AM, John Aherne johnahe...@rocs.co.uk wrote: This is a really basic problem we are trying to decide about, We have programs that run quite happily, so far. Its main task is to receive data from port A and send it out via port B. Then receive data via port B and send it out via port A. It's pretty much like a chat setup. You just build up a list of connected clients and send data to them as required One side A receives some input from a tcp port - about 100-200 characters, and forwards it to another port B. We do not need to wait for any response. If we get a response we pick that up through line receiver. We also run a calllater to check if we got a response on linereceiver within the timeframe specified. If not we drop the connection. Traffic coming in from port B is analysed and some subset is sent back to port A. Ignoring port A for the moment, just concentrating on port B, we have tried three options:-- 1. We set up a defer to handle the sendline to port B so that the reactor would schedule it in its own good time. No threads involved using the standard twisted setup. When we get a response through receiveline we fire the callback defer. If we timeout via callLater we fire the errback to clear the defer. In this case the defer does not seem to be doing very much 2. Now a fresh pair of eyes is looking at the code and saying why are we using a deferred for sending data to port B. We could just issue a straight sendline as part of the main code and carry on. If we get a response via linereceiver,we process it normally, otherwise we set our callLater running and timeout and lose the connection. So no deferreds required at all. It does seem to work.What we are not sure about is what penalty is incurred in terms of reliability or throughput by using sendline without a deferred. We are not too sure what the holdup will be and whether it could end up halting the show. Is it better to schedule these messages via deferreds or am I missing something obvious 3. So we then did an experiment and used defertothread to run the sendline in a separate thread with its own defer to maximise the asynchronous running of the code. So now we are running threads when one of the reasons for looking at twisted was that we could avoid threads as much as possible. The conundrum we are trying to resolve now is which option should we use. Do any of the options have a built-in problem awaiting the unwary. In theory all 3 options work. But if No 1 works well enough for our volume of traffic should we adopt that one. Or is it better to start using the defertothread option. Is there a simple answer The traffic is not large, upto a 100-200 remote devices on port B. They will send GPS data every 20 secs, and about 500 messages of about 200 bytes average throught the day. The remote devices will respond in an irregular manner without dropping the connection, so we force a disconnectf if important messages are not getting through. They are then forced to reconnect. We have looked through the code searching for enlightment and it does seem to be well documented, but the information we are looking for comes well before the doc strings. Hopefully, someone can give us some pointers in the right direction. Thanks for any help. John Aherne It seems to me that the volume of traffic you are dealing with isn't so high that you need to worry too much about direct sendline causing problems. If I were writing this from scratch based on my understanding of what you've written above, I would probably go with option 2. (Keep in mind, my understanding may be flawed...so...) However, if you've already got things working with option 1, and the added complexity isn't causing you any trouble, I don't see any real reason not to use that, since you've already got that working. Others may disagree... Option 3 seems totally unnecessary to me. I typically stay away from threads in Twisted unless I have a long running non-network process to deal with (disk access, db access, heavy math processing, etc.). Especially because of the relative heaviness of threads when using Python (due to complex interactions with the GIL), I would avoid this method...it will probably hurt performance more than Option 1 (though still probably not enough to matter). Others feel free to slap me if I'm giving bad advice :) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] My first wxPython App with Twisted
)
self.Show(True)
self.Bind(wx.EVT_CLOSE, self.OnCloseWindow)
self.Bind(wx.EVT_ICONIZE, self.OnIconify)
def on_login_click(self, username, password):
self.Hide()
def OnCloseWindow(self, event):
if self.tbicon is not None:
self.tbicon.Destroy()
event.Skip()
def OnIconify(self, event):
self.Hide()
def get_image(self):
img= wx.Image(opj('client.ico'), wx.BITMAP_TYPE_ICO)
return img
def get_icon(self):
img = self.get_image()
icon = wx.IconFromBitmap(img.ConvertToBitmap() )
return icon
if __name__ == '__main__':
main()
--
If somebody can help me out, I would be much obliged.
Well, it's hard to say without looking at your Twisted code, but what
exactly is the LoopingCall for? It seems like it would be run regardless of
whether the user had logged in or not.
I'm guessing, but maybe client.run() shouldn't be run until you have a
username and password?
Might be a place to start looking, anyway. It looks like you've set things
up properly in main() an App.OnInit(), but I haven't worked with Twisted +
wx in a while (pre-wxreactor), so I'm not 100% sure...
I guess it boils down to what exactly client.run() does...
Kevin Horn
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted and wxPython
There's also a wxpython example in the twisted subversion repository. See: http://twistedmatrix.com/trac/browser/trunk/doc/core/examples/wxdemo.py It follows the basic pattern that Godson provided. Kevin Horn 2009/8/28 陶艺夫 artman...@gmail.com Thanks. I'll try it out. 2009/8/28 Godson Gera godso...@gmail.com from twisted.internet import wxreactor wxreactor.install() from twisted.internet import reactor . write your code here . reactor.registerWxApp(app) reactor.run() You don't need wx mainloop. The key thing here is to import wxreactor first and install it and then import reactor. 'app' in the last but one line is the instance of your wx.App . On Fri, Aug 28, 2009 at 11:05 AM, 陶艺夫 artman...@gmail.com wrote: Hi, Anyone can give me a sample app skeleton to get two frameworks work together well? Or just give me a link where I can figure out the technical details. Thanks. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- Thanks Regards, Godson Gera http://godson.in ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] network server on multiprocessor hardware
On Sat, Sep 5, 2009 at 9:07 AM, Bokhan Artem a...@academ.org wrote: Laurens Van Houtven пишет: Are you sure it's dying out because of CPU limitations? Yes, with 1k req/s cpu is 100% busy with python. Can you show any real applications where this happens? I do not have real application in python/twisted now. That could be any internet server with high load or under dos. In my experience your network IO always gives up ages before my processors croak :-) It depends. I happen to be in the middle of writing a project with a big computational component (at least when I'm not hacking away at twisted.positioning), and generally the heavy computational stuff doesn't live in the Twisted layer. I do not have any heavy computations, just high request rate. If you are looking for load balancing, you might also find txLoadBalancer useful: http://groups.google.com/group/txlb-dev Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Structural issues in Lore XHTML documents
I've been looking over the xhtml documents used to generate the twisted documentation, and I've noticed a number of issues: - some docs do not have a DOCTYPE declaration, I think they should all have one - of those documents that do have DOCTYPEs, some are using xhmtl-strict, and some are using xhtml-transitional, which is preferred? I think they should all use the same one - some of the docs are lacking an xml namespace attribute in their root html element...I think they should either all have one, or none of them should - according to: http://twistedmatrix.com/projects/lore/documentation/howto/lore.html all of the docs should have the same text in both their title element and their h1 element...this is not the case I'm planning to correct some of these issues, but I wanted to get others' opinions on exactly how to go about it. For example, what DOCTYPE to use, XMLNS to use, etc. Thoughts? Suggestions? I've opened a ticket (#4050) at http://twistedmatrix.com/trac/ticket/4050 Please comment. Thanks, Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Structural issues in Lore XHTML documents
On Mon, Oct 5, 2009 at 1:25 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Mon, Oct 5, 2009 at 12:22 PM, Kevin Horn kevin.h...@gmail.com wrote: I've been looking over the xhtml documents used to generate the twisted documentation, and I've noticed a number of issues: Are these issues really affecting you in some way? Yes and no. I'm playing around with some automated processing of these documents (mostly as an exercise to learn lxml), and I'm having to work around some of these things. And since I was thinking about it anyway, I thought I'd take the opportunity to fix up some broken windows. I'm planning to correct some of these issues, but I wanted to get others' opinions on exactly how to go about it. For example, what DOCTYPE to use, XMLNS to use, etc. They are supposed to be XHTML, so they should be annotated as appropriate for that format. Lore doesn't define its own attributes, on purpose: it uses existing XHTML facilities like class to encode its metadata. The main thing I was worried about was whether they were supposed ot be xhtml-strict or xhtml-transitional. Thoughts? Suggestions? I'm sure we'd be happy to accept some patches to clean these documents up, especially if it's bothering you, but it seems pretty low-priority to me :). It's not bothering me per se, and we're agreed that it's not any kind of priority, just thought I might as well try and fix it up while I was looking at it. FYI: I've hacked together a simple lorelint script to automatically check for these type of issues. Happy to share if anyone thinks it might be useful for future release mgmt automation or whatever. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Structural issues in Lore XHTML documents
On Mon, Oct 5, 2009 at 1:44 PM, Kevin Horn kevin.h...@gmail.com wrote: FYI: I've hacked together a simple lorelint script to automatically check for these type of issues. Happy to share if anyone thinks it might be useful for future release mgmt automation or whatever. For benefit of future readers...Lore already does something like this. Try: lore -olint mydocument.xhtml Thanks to exarkun for pointing this out. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] getting #2015 (Generic SASL library) moving again?
On Mon, Oct 5, 2009 at 1:49 PM, Kevin Horn kevin.h...@gmail.com wrote: On Mon, Oct 5, 2009 at 1:32 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Mon, Oct 5, 2009 at 12:34 PM, Kevin Horn kevin.h...@gmail.com wrote: I'd like to see this ticket get moving again. Excellent! I've gone over what's been done so far (and put my comments in the ticket), and it looks like everything is at least mostly ready to be merged into the Twisted trunk. Your analysis on the ticket looks fairly thorough. Thanks! Though, really I just went over the changesets that therve submitted, and checked them against exarkun's suggestions. So what else needs to be done to get this to happen? I'm happy to try and get it done, but a little direction would be really handy. ;) The workflow is pretty simple. 1. author submits change 2. reviewer reviews change 3. someone responds to feedback 4. someone becomes author, repeat Right now we're stuck in the middle of step 3, but given that therve does not appear to be actively working on this branch, the path forward is clear. *You* become someone, and submit a patch that finishes dealing with the issues that you have mentioned as not already taken care of by therve's subsequent changesets. The best way to do this is to submit a patch against the branch, rather than against trunk. Given such a patch, a twisted committer can then apply the patch to the branch and submit to buildbots (etc) as appropriate. By submitting it against the branch, you make it easy for the reviewer to examine the changes you're submitting in isolation from the rest of the larger change, to see if they address the points that you've brought up. If the reviewer is happy with your changes, then the patch + branch will be immediately merged to trunk; of course, the next review might have a little more feedback for you to deal with ;). Thanks for your interest! That's kind of what I figured. From my previous review of the current status, it looks pretty well done to me, but I'll start taking a look at it in more detail, and see what I can come up with. Might need some more tests, etc. Kevin Someone Horn FYI, this ticket back up for review... Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Can browse, but getPage() fails on localhost address
On Thu, Oct 29, 2009 at 12:36 PM, Steve Steiner (listsin) list...@integrateddevcorp.com wrote: On Oct 29, 2009, at 1:10 PM, Maarten ter Huurne wrote: On Thursday 29 October 2009, Steve Steiner (listsin) wrote: I can browse to either URL in any browser, getPage() works fine on the external address, 404's on the localhost address. Does the same thing happen with 127.0.0.1 instead of localhost in the URL? I once encountered a nasty setup where a machine itself did not have localhost in /etc/hosts and the DNS server returned the IP address of another machine on the network... Yes, unfortunately, same thing and the hosts file is also correct. This is in OS X, and the valida†or URL I'm calling is being served by Apache 2.2, and is a ScriptAlias like so (in an includ of httpd.conf): ScriptAlias /w3c-validator/check '/Applications/Validator- SAC.app/Contents/Resources/validator/httpd/cgi-bin/check_wrapper.sh' Thing that's got me baffled is that I can go to it just fine in a browser, I can `curl` it, but, now, after a little more digging, wget can't get it either! So: curl http://localhost/w3c-validator/check -- returns HTML code any browser to http://localhost/w3c-validator/check -- returns HTML code t.w.c.getPage(http://localhost/w3c-validator/check;) -- 404 wget http://localhost/w3c-validator/check -- 404 So what the heck is the difference in how these things resolve that address!? Thanks, S When you say resolve that address do you mean the IP Address or the URL? If the requests are in fact going to the same address (IP), I would take a look at the HTTP headers for each method and compare them. Wget failing in a similar fashion to getPage() may actually help you diagnose this. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted and wxPython
On Thu, Nov 5, 2009 at 12:14 PM, Jon Mills jonmi...@moosesoup.com wrote: Can anybody point me towards a nice up-to-date example of the use of Twisted with wxPython? I assume (hope) there's some way to bind the wx event loop into Twisted's Reactor? Or do they need to run in separate threads/processes? Thanks in advance, Jon Mills I typically base my code the example that comes with Twisted. http://twistedmatrix.com/trac/browser/trunk/doc/core/examples/wxdemo.py Is this not working for you, or had you just not seen it? (It took me quite a while to find... ;) ) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Caching mechanism
On Mon, Nov 9, 2009 at 4:47 PM, Paul Hubbard hubb...@sdsc.edu wrote: Both of the links from pypi are dead, alas. Do you know if that project is alive somewhere else? Cheers, Paul There's something that's probably similar here: http://code.activestate.com/recipes/498245/ Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Proposal (almost) to switch from Lore to Sphinx
On Thu, Nov 12, 2009 at 7:01 AM, exar...@twistedmatrix.com wrote: On 05:22 am, gl...@twistedmatrix.com wrote: On Nov 11, 2009, at 7:37 PM, Kevin Horn wrote: You're correct that lore cannot include partial external files. Actually, it has limited support for this. The PB documentation relies on it (search for skipLines in doc/core/howto). And if it were implemented better, I think I would find it a useful feature. Jean-Paul Updated in the proposal. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Multi-reactor architecture
On Thu, Nov 12, 2009 at 12:53 PM, vit...@synapticvision.com wrote: You right, distributed systems architecture. Probably I need to rephrase my question more precisely: how to build distributed system architecture with Twisted technology only ? As you mentioned, even reverse proxy could be a another twisted process. You might look at txLoadBalancer: https://launchpad.net/txloadbalancer Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Proposal (almost) to switch from Lore to Sphinx
On Thu, Nov 12, 2009 at 11:36 AM, Kevin Horn kevin.h...@gmail.com wrote: On Wed, Nov 11, 2009 at 11:22 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: Release early, release often. Most efforts like this end up moribund: you sound serious right now, but never underestimate the extent to which life will get in the way. If you put up what you've got somewhere public ASAP, (A) some people will probably show up to egg you on, (B) it's possible for them to contribute (and those contributions will be a motivation on days when this seems like a ridiculous amount of effort just to make Twisted's docs look nicer), and (C) they can pick up the slack if you need to disappear, either temporarily or permanently. I'll try to put up what I've got later today. OK, finally got the current results of the conversion process up. Sorry for the delay, my weekend got hijacked by the Swine Flu. Some of that life getting in the way, I guess. I want to reiterate that this is _not_ even _close_ to finished. But it's what I've got so far. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Proposal (almost) to switch from Lore to Sphinx
On Tue, Nov 17, 2009 at 4:00 AM, lasizoillo lasizoi...@gmail.com wrote: 2009/11/11 Kevin Horn kevin.h...@gmail.com: I have written up a proposal here: http://twistedsphinx.funsize.net/proposal.html In this proposal you say: Development Work * fabric fabfile for automation of docs workflow Future Work * This page has some stuff on using cog in RestructuredText comments to generate the output of sample scripts. The second point (in Future Work) uses cog that is integrated in paver: * Paver allow automation of docs workflow (fabric is not needed). * With paver you can create an bootstraped instalation: http://www.blueskyonmars.com/projects/paver/getting_started.html#but-people-don-t-have-paver-yet * Paver is compatible with setuptools and distutils. I think that fabric is great for cluster distributions, but paver is better for manage docs. Regards, Javi For the moment, I plan to continue using fabric, for the following reasons: - I'm already familiar with it - it does what I need (run local commands, and send stuff over SSH) I think Paver might be very useful down the road, though. I'll definitely check into it more as this project progresses. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] tee method in twisted.internet.defer.py ?
On Tue, Nov 17, 2009 at 10:27 AM, Terry Jones te...@jon.es wrote:
JP == exarkun exar...@twistedmatrix.com writes:
I don't need it committed, I was just wondering why something like that
doesn't already exist. Re patterns: it would be great to have a series of
diagrams illustrating situations one might want to deal with and showing
how these can be done with Deferred code. Your Epsilon class could be shown
as a hub with new listeners arriving, ready for the triggering of the
callback. The tee diagram would show a T (or multiple Ts on the same
callback chain) in the obvious way. add{Callback,Errback} is like a normal
UNIX pipe and, stretching things a little, chainDeferred reminds me a bit
of UNIX exec (though with real differences).
There's a PyCon talk in there somewhere. Like Patterns of Asynchronous Flow
of Control using Twisted Deferreds. It often feels to me like there's a
lurking taxonomy of Deferred use cases or building blocks that's just
waiting for someone to come along and make really clear to the world. I'd
really love to do it, but I just don't have the time.
I would so love to see that talk...or read that paper...or whatever.
Kevin Horn
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report
On Mon, Nov 30, 2009 at 4:39 PM, Marco Giusti marco.giu...@gmail.comwrote: On Mon, Nov 30, 2009 at 03:05:30PM -0600, Kevin Horn wrote: [...] Thanks for listening! some notices about what i get[1]. mainly i'd like if you could make the documentation more readable when the browser's window is not really big. usually i do not use the browser with the window maximized so will be really helpful if you can check the following points: * in the navigation bar twisted web documentation is repeated twice and is breaked in two lines. * the table of contents steals a lot of vertical space while it is visible only at the top of the page. in this circumstances i prefer the lore way, ie. showing the table of contents in the normal content flow at the top and not in a side column. * again the left blue column steal vertical space an maybe can be reduced. * would be great if the layout will fit in the page width: as you can see the index link is half hidden. obiviusly with a bigger window all the aboves are not more valid and all looks pretty. ciao m. [1] http://img44.imageshack.us/img44/1953/sphinx.png Marco, Thanks for the feedback. I really haven't worked at all on formatting/theming the docs as of yet (what you see is just a theme included with Sphinx), but I will certainly keep your comments in mind when I get to that point. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report
On Wed, Dec 2, 2009 at 8:38 PM, exar...@twistedmatrix.com wrote: On 2 Dec, 11:04 pm, kevin.h...@gmail.com wrote: [1] http://twistedsphinx.funsize.net/proposal.html [2] http://twistedsphinx.funsize.net/index.html Any of the core Twisted devs care to sound of regarding the proposed timelines? Silence implies consent. Or in this case approval... :) Timeline might not be the right word, since as far as I can tell there's little that's actually about time there. :) Let me know if I overlooked something. Well, timeline was the word Glyph used previously for order in which stuff will be done. So that's what I stuck with. As far as the (I'm going to call it) roadmap goes, the thought that's pushed its way to the fore for me is that I'd like to try this with something smaller and simpler than Twisted first. It would be nice if the Divmod projects would qualify here, but they may not be active enough for any real experience to accumulate. I haven't run my lore2sphinx script against the Divmod stuff yet, but I could certainly try it. The intention is to convert the Divmod stuff as well as the Twisted stuff eventually though. I've been concentrating on the Twisted docs, since they seemed the highest priority, but I could detour a bit if you like. Obviously that would slow progress on the Twisted stuff a little. pyOpenSSL might work. It uses the original CPython tex documentation system, so the conversion would (presumably) be done with whatever tools were used for CPython's own documentation, not the new Lore-Sphinx converter. It would still be a useful way to get more familiar with Sphinx, though. The downside is that I'm pretty much the only Twisted developer who works on pyOpenSSL, so it wouldn't help anyone else get any experience. I have no idea what tools were originally used for that conversion. Might be interesting to try if we could find out though. Also, I presume we'd need the sources for the docs. Are they in the tarball? Or elsewhere? Of course if your objective is to learn Sphinx, you might be better off converting the pyOpenSSL docs by hand. They seem pretty short (except for section 3). If you'd like to try that, I'd be happy to assist. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report
On Thu, Dec 3, 2009 at 10:11 AM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Dec 3, 2009, at 10:06 AM, exar...@twistedmatrix.com wrote: I want some experience with Sphinx so that can verify the premise that Sphinx is better than Lore. So, the experience isn't necessary for the conversion, it's necessary for me to be comfortable that the conversion is a good idea. Oh. I am definitely taking that as a given. Or rather: having frequently interacted with the output of both sphinx and lore, I was already confident that it was better than Lore in most ways before we began this conversation. I had a few specific misgivings, which Kevin has since addressed. Sphinx clearly has the *features* that we need; Kevin's analysis looks pretty thorough, and even the current error-laden converted documentation serves as a reasonable proof of concept. Plus, the plan specifically includes resolving the remaining issues with that documentation as a prerequisite to getting it merged. So while I could believe that there will be bad things about Sphinx which may surprise me, I find it hard to believe that I could be surprised by any bug in Sphinx so awful that it would be *enough worse* than Lore to make continuing to maintain Lore a preferable option, given the presence of a documentation maintainer willing to do the drudgework of conversion. That said, it might be a good idea to do the Divmod project conversions first for other reasons. They have less documentation, so there's less to review, and the process could be completed more quickly, letting us get some benefit out of the process earlier on. Since these projects are the only known remaining users of lore, we could get rid of lore immediately when twisted itself is converted, rather than letting it linger until its users have converted. I think maybe I'll try to do a quickie conversion on one of the Divmod projects (probably Nevow) in the next couple of days. I'm not planning on doing a theme or anything yet...just the content conversion, followed by manual cleanup. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Lore to Sphinx Conversion Progress Report 2
Hmmm...I guess my send an update every week idea was a little optimistic. :) Again haven't had as much time to devote to this as I would have preferred. More of Glyph's real life getting in the way I guess (had a new baby added to the household last week!), but here's a quick rundown: - proposal [1] once again updated, though not much...timeline for Phase 0 still needs some work - lore2sphinx tool: some progress, though I'd hoped for more. Inclusion of external files (like code samples) now works, though I'm borking leading whitespace somewhere, so python files are mis-formatted. Some of the whitespace handling is improved (nested lists seem to work OK now), but it needs more work. I think I have a general plan for how to fix this, but it just requires some elbow-grease. I've also done quite a bit of refactoring, to make my mess of code a bit easier to grok. Still to do: tables, citations, footnotes, and some improved handling of code tags, as well as the afore-mentioned whitespace-handling fixes. - theme: haven't really started this yet, just a bit of research - Divmod docs: haven't really touched this yet...it's in the queue - PyOpenSSL docs: I got an email from Georg Brandl, with a pointer to the tools used to convert the old LaTeX Python docs to Sphinx [2]. According to Georg, it may need some tweaking, but should pretty much work for converting the pyOpenSSL docs. Looking at the code for this has also helped me a bit with a few minor problems I was having. So kudos for Georg! Also, it looks like the source of the pyOpenSSL docs are indeed in the source tarball, so hopefully I can make some headway on this pretty soon. I'm not really speeding along with this project, but I am making some slow progress. As the tortoise said: Slow and steady wins the race! Comments and suggestions welcome! [1] http://twistedsphinx.funsize.net/proposal.html [2] http://svn.python.org/projects/doctools/converter ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 2
On Fri, Dec 18, 2009 at 1:40 AM, Matt Bone thatmattb...@gmail.com wrote: I'm so sorry, I was trying to express interest in the goings on but my damn cat walked on the keyboard and sent a blank message. If I'd been drinking milk when I read this, it would have come out my nose. The same thing has actually happened to me. :) I'm handy around a text editor/document conversion scripts/sphinx, so please let me know if there are any simple tasks I can assist with. Also, I registered for pycon this evening, so please say hello if you bump into me. Take care, --matt Thanks for the offer, I'll probably take you up on it at some point. If you (or anyone) would like to go ahead and start work on a Sphinx theme for the Twisted docs, I think that would be easy to work on in parallel to my own efforts. As far as the actual conversion, I think I have ideas for my remaining issues, it's just a matter of finding the time to work on them. I expect another set of eyes would be useful once I get things into a more understandable state though, if you'd like to volunteer for that. Sadly I probably won't make it to PyCon this year, though I haven't _entirely_ ruled out the possibility. Which really sucks, as it looks like it will likely be the best ever. :( Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 2
On Thu, Dec 17, 2009 at 11:40 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Dec 17, 2009, at 4:00 PM, Kevin Horn wrote: Hmmm...I guess my send an update every week idea was a little optimistic. :) Again haven't had as much time to devote to this as I would have preferred. More of Glyph's real life getting in the way I guess (had a new baby added to the household last week!), but here's a quick rundown: Wait, *my* real life is getting in the way? I would have figured that if my real life were getting in *both* of our ways, I'd be getting a lot more done! ;-). I was referring to the Lefkowitz Definition of Real Life(tm): Real life: N. That which interferes with the generation of nifty open-source code. :) Thanks for continuing on with this project, and continuing to write these progress reports. I'm really looking forward to seeing this plan get executed! Sadly I don't have any feedback for you because it's hard to tell what's changed since the last time. Do you have a change log or diffs of some kind somewhere? Well, I've got the code (and proposal) in an SVN repos, but it's not public, and I can't really make it so (long story). I've got an item on my TODO list to extract it from there and put it up someplace else, but I haven't gotten to it yet... In the meantime, I can post a diff of the proposal if you like, or a part of my SVN log, but you really haven't missed anything significant on the proposal, I'm afraid. Mostly just a minor cleanup. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 2
On Fri, Dec 18, 2009 at 11:23 AM, Kevin Horn kevin.h...@gmail.com wrote: On Fri, Dec 18, 2009 at 1:40 AM, Matt Bone thatmattb...@gmail.com wrote: I'm so sorry, I was trying to express interest in the goings on but my damn cat walked on the keyboard and sent a blank message. If I'd been drinking milk when I read this, it would have come out my nose. The same thing has actually happened to me. :) I'm handy around a text editor/document conversion scripts/sphinx, so please let me know if there are any simple tasks I can assist with. Also, I registered for pycon this evening, so please say hello if you bump into me. Take care, --matt Thanks for the offer, I'll probably take you up on it at some point. If you (or anyone) would like to go ahead and start work on a Sphinx theme for the Twisted docs, I think that would be easy to work on in parallel to my own efforts. As far as the actual conversion, I think I have ideas for my remaining issues, it's just a matter of finding the time to work on them. I expect another set of eyes would be useful once I get things into a more understandable state though, if you'd like to volunteer for that. Sadly I probably won't make it to PyCon this year, though I haven't _entirely_ ruled out the possibility. Which really sucks, as it looks like it will likely be the best ever. :( Kevin Horn Other tasks that can help move this forward, but that don't require work on the actual conversion code: - identify branches which have lore patches in them - identify tickets which propose to fix/modify Lore functionality - identify wiki pages that need to be changed, and the necessary changes - make suggestions on the overall conversion process, as documented in the proposal. If anyone is interested in helping out, just say so here, or drop me a line off-list. I'm also happy to send a copy of the code I have so far to anyone who's interested...just send me an email off list, and I'll send you a tarball. And if anyone has suggestions about where/how to share the code someplace, I'm happy to listen to those too. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 2
On Fri, Dec 18, 2009 at 2:51 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Dec 18, 2009, at 3:37 PM, Kevin Horn wrote: I don't currently have access to Twisted SVN (as far as I know, anyway), but if the Twisted guys want to give me access, I'm fine with that. I'm not particularly opposed to it, but at this stage I don't think it would help very much either. The sandbox is a crummy place to stage stuff. Otherwise, BitBucket is probably the best option, though it'll be a little bit of a learning curve for me. I've used Mercurial a little bit, but never for a real project, and I've never used BiBucket (though I think I have an account, for some reason). We already have an active mirror of Twisted on launchpad https://code.launchpad.net/twisted, which might be easier to get started with if you want to make a branch for the *output* of the conversion. lore2sphinx itself could probably live anywhere, though. I think I'll probably end up putting lore2sphinx up on BitBucket...when I get around to it. Hopefully I'll get around to doing this in the next week or so, but with the holidays coming up, I wouldn't hold my breath. ;) I don't see a whole lot of point in putting the output into any sort of version control at this point, since the idea is not to edit it at this stage. I'll continue to put up my results at http://twistedsphinx.funsize.net periodically just for people to look at, though. Once the automated conversion part of this is done, it will make sense (and indeed be necessary) to VC the Sphinx sources, though. BTW...footnotes work now. :) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 2
On Fri, Dec 18, 2009 at 4:51 PM, Mikhail Terekhov ter...@gmail.com wrote: On Fri, Dec 18, 2009 at 4:45 PM, Kevin Horn kevin.h...@gmail.com wrote: I don't see a whole lot of point in putting the output into any sort of version control at this point, since the idea is not to edit it at this stage. I'll continue to put up my results at http://twistedsphinx.funsize.net periodically just for people to look at, though. May be then attach it to some ticket in Trac? I could, but I don't see a whole lot of point at this stage, since the output changes every time I update and run my conversion script. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 2
BTW, lore2sphinx and accompanying Sphinx skeleton project now up on bitbucket here: http://bitbucket.org/khorn/lore2sphinx/ Thanks to ssteinerX for guiding me through the process. Enjoy! Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 2
On Sun, Dec 27, 2009 at 10:50 AM, exar...@twistedmatrix.com wrote: On 22 Dec, 10:15 pm, kevin.h...@gmail.com wrote: BTW, lore2sphinx and accompanying Sphinx skeleton project now up on bitbucket here: http://bitbucket.org/khorn/lore2sphinx/ Thanks to ssteinerX for guiding me through the process. I was reminded of another aspect of the conversion which needs some consideration, internationalization and localization. I assume there are tools for internationalizing sphinx documentation. That just leaves localization. There is currently an effort underway to localize the Twisted documentation to Chinese (simplified) and possibly a couple other languages: https://translations.launchpad.net/twisted I have no idea how one would go about converting this data to account for a change in the source format of the docs. Does anyone else? Jean-Paul Sphinx does have some translation support, though I confess I haven't looked at it. It uses the Babel package. No idea how to convert the launchpad translations at the moment, but one thing at a time, eh? :) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Lore to Sphinx Conversion Progress Report 3
Hello again, and welcome to another update on the Twisted Lore to Sphinx conversion project. Current Sphinx results are in the usual place at: http://twistedsphinx.funsize.net/ Here's what's new: - The lore2sphinx conversion script is now up on bitbucket. You can find it here: http://bitbucket.org/khorn/lore2sphinx/ . Special thanks to Steve Steiner (ssteinerX) for helping out a bitbucket newbie. - lore2sphinx now reads input and output directories from a config file. The config file can have one or more profiles, each of which contains config info for a set of documentation. At present, the script just uses the default profile (set in the config file), but I'll add a command line switch to specify the profile Real Soon Now (tm), which will make it easy to use a single config file for multiple projects (e.g. twisted, nevow, mantissa, etc.) - lore2sphinx has also been refactored a bit, which solves a lot of whitespace handling issues. The Sphinx build now gives fewer than 100 errors (currently 65, started off in the 500-600 range)! Hooray! Still, not all of the output is fully correct, and some tags (notably tables) are still unhandled. - I still haven't started any work on a theme...I'll get to it eventually, unless someone else does first. Any volunteers? - Divmod docs: new config file support is a step in this direction, look for something soon... - PyOpenSSL docs: kind of a sideline to this so I haven't really moved forward on this yet, but I will eventually If anyone out there would like to help out, I can certainly use the assistance. Lore2sphinx could probably use some extra eyeballs and brains looking at it, thought the basic structure is pretty much laid out. Any ideas/comments/criticisms welcome. If you're intimidated by the idea of working on the actual lore2sphinx conversion script (or just don't want to), here's some other stuff that needs doing: - work on a twisted (or Divmod) Sphinx theme which at least mostly matches the website (or the Divmod website) - identify branches which have lore patches in them - identify tickets which propose to fix/modify Lore functionality - identify wiki pages that need to be changed, and the necessary changes - make suggestions on the overall conversion process, as documented in the proposal. I'm sure I'll manage to get all this done eventually, but with more people, it will get done sooner. And sooner is better. :) If you'd like to help out with one (or more) of these tasks, just give a shout here, or email me off-list. Thanks! Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 3
On Wed, Dec 30, 2009 at 2:40 PM, Thijs Triemstra | Collab li...@collab.nlwrote: Hi Kevin, On 30 Dec 2009, at 21:05, Kevin Horn wrote: If you're intimidated by the idea of working on the actual lore2sphinx conversion script (or just don't want to), here's some other stuff that needs doing: - work on a twisted (or Divmod) Sphinx theme which at least mostly matches the website (or the Divmod website) - identify branches which have lore patches in them - identify tickets which propose to fix/modify Lore functionality I can work on identifying those tickets and branches in Twisted, but should I put that info in a ticket or on a Twisted wiki page? And should the tickets with Lore enhancements be closed immediately or listed first? I'm open to suggestions as to where to list the tickets. A Trac ticket or a wiki page would certainly work, or we can integrate the list into the proposal document. What might be a good solution though is to just add a Trac keyword to all of the relevant tickets, or even just identify a specific search for them in Trac (is it enough to just do a custom search for Lore component tickets?). I think we should just come up with a list as a first step. Though I really hope to complete the conversion before too much longer, you never know what could happen, and if it gets delayed significantly, it might be useful to still have those tickets open. Thanks for all your work! Cheers, Thijs ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 3
On Wed, Dec 30, 2009 at 3:00 PM, Christopher Armstrong ra...@twistedmatrix.com wrote: Perhaps just add a label 'lore-changes' to all the tickets which do so. Ack! Chris beat me to it! Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 3
On Wed, Dec 30, 2009 at 2:05 PM, Kevin Horn kevin.h...@gmail.com wrote: Hello again, and welcome to another update on the Twisted Lore to Sphinx conversion project. Current Sphinx results are in the usual place at: http://twistedsphinx.funsize.net/ Here's what's new: - The lore2sphinx conversion script is now up on bitbucket. You can find it here: http://bitbucket.org/khorn/lore2sphinx/ . Special thanks to Steve Steiner (ssteinerX) for helping out a bitbucket newbie. - lore2sphinx now reads input and output directories from a config file. The config file can have one or more profiles, each of which contains config info for a set of documentation. At present, the script just uses the default profile (set in the config file), but I'll add a command line switch to specify the profile Real Soon Now (tm), which will make it easy to use a single config file for multiple projects (e.g. twisted, nevow, mantissa, etc.) - lore2sphinx has also been refactored a bit, which solves a lot of whitespace handling issues. The Sphinx build now gives fewer than 100 errors (currently 65, started off in the 500-600 range)! Hooray! Still, not all of the output is fully correct, and some tags (notably tables) are still unhandled. - I still haven't started any work on a theme...I'll get to it eventually, unless someone else does first. Any volunteers? - Divmod docs: new config file support is a step in this direction, look for something soon... - PyOpenSSL docs: kind of a sideline to this so I haven't really moved forward on this yet, but I will eventually If anyone out there would like to help out, I can certainly use the assistance. Lore2sphinx could probably use some extra eyeballs and brains looking at it, thought the basic structure is pretty much laid out. Any ideas/comments/criticisms welcome. If you're intimidated by the idea of working on the actual lore2sphinx conversion script (or just don't want to), here's some other stuff that needs doing: - work on a twisted (or Divmod) Sphinx theme which at least mostly matches the website (or the Divmod website) - identify branches which have lore patches in them - identify tickets which propose to fix/modify Lore functionality - identify wiki pages that need to be changed, and the necessary changes - make suggestions on the overall conversion process, as documented in the proposal. I'm sure I'll manage to get all this done eventually, but with more people, it will get done sooner. And sooner is better. :) If you'd like to help out with one (or more) of these tasks, just give a shout here, or email me off-list. Thanks! Kevin Horn Another quick update... - lore2sphinx now accepts a command line parameter to specify which config profile to use - added stub sphinx projects for Divmod Nevow, Mantissa, and Epsilon. I've converted the docs for these projects and built them (mostly) successfully, but I haven't published them anywhere yet (coming soon). Have I missed any other Divmod projects with Lore docs? Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] lore2sphinx table handling?
Sphinx uses plain ReST tables, and from what I can tell, ReST has 4 (yes, 4!) ways of marking up tables: Grid tables http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#grid-tables - ASCII-art style tables Simple Tables http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#simple-tables - A different style of ASCII-art table, simpler but less powerful than Grid Tables CSV Tables http://docutils.sourceforge.net/docs/ref/rst/directives.html#csv-table - data is provided in CSV format in the body of a ReST directive List Tables http://docutils.sourceforge.net/docs/ref/rst/directives.html#list-table - data is provided as nested lists in the body of a ReST directive My question is, what format should the lore2sphinx tool target? Any of these formats should work fine, but I'm curious as to what people (in particular the core devs) think as to which should be the preferred method in the Sphinx documentation. Keep in mind that there are only 2 tables in all of the Twisted docs, and one of them is in the documentation for Lore, which (hopefully!) should be going away once this project is completed. So it's not like it would be a whole lot of work to change the preferred format later. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] lore2sphinx themeing
Thanks to a quick patch from Michael Thompson, the current output of the Sphinx conversion [1] now matches the color scheme and typographic style of the main twisted site. It's using the layout of the 'default' sphinx theme (previously was using the 'sphinxdoc' theme). I'd like to know what people out there want/expect the final theme of the Sphinx version of the Twisted docs to look like. The floor is open...please offer your suggestions! [1] http://twistedsphinx.funsize.net/ ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Weekly Bug Summary
I'd like to point out that for three of the last four weeks, more tickets have been closed than have been opened. Go team! (especially exarkun and jesstess). Kevin Horn On Sat, Jan 9, 2010 at 11:05 PM, exar...@twistedmatrix.com wrote: Bug summary __ Summary for 2010-01-03 through 2010-01-10 Bugs opened: 6Bugs closed: 7 Total open bugs: 1217 (-1) |== Type Changes |== Priority Changes |== Component Changes |Defect: -2 |Highest: -1 |Conch: +0 |Enhancement: +1 |Normal: -1 |Core:+1 |Low: +1 |Mail:+0 |Release Management: -1 |Trial: -1 Total TicketsOpen Tickets New / Reopened Bugs __ = Normal = [#4197] t.conch.insults.window.ScrolledArea is not instantiable. (opened by ali) defect conch http://twistedmatrix.com/trac/ticket/4197 [#4199] spawnProcess should reset signal handlers to default before exec()ing new executable (opened by itamarst) enhancement core http://twistedmatrix.com/trac/ticket/4199 [#4200] SMTP Client tutorial uses bad markup (opened by khorn) (CLOSED, fixed) enhancement mail http://twistedmatrix.com/trac/ticket/4200 [#4201] Broken link in core/howto/tutorial/intro (opened by khorn) (CLOSED, fixed) enhancement core http://twistedmatrix.com/trac/ticket/4201 [#4202] Bad markup in doc/web2/howto.object-traversal (opened by khorn) enhancement core http://twistedmatrix.com/trac/ticket/4202 = Low = [#4198] twisted.protocols.sip assumes DelayedCalls are scheduled by wallclock time (opened by ivank) defect core http://twistedmatrix.com/trac/ticket/4198 Closed Bugs __ = Highest = [#3251] Web2 tarball install doesn't jive with setuptools of integrated tarball (opened by ralphm, closed by thijs, wontfix) defect release management http://twistedmatrix.com/trac/ticket/3251 = Normal = [#3848] twisted.test.test_randbytes.ConditionalSecureRandomTestCase fails on Python tr...@head (opened by ivank, closed by ivank, fixed) defect core http://twistedmatrix.com/trac/ticket/3848 [#4200] SMTP Client tutorial uses bad markup (opened by khorn, closed by jesstess, fixed) enhancement mail http://twistedmatrix.com/trac/ticket/4200 [#4201] Broken link in core/howto/tutorial/intro (opened by khorn, closed by jesstess, fixed) enhancement core http://twistedmatrix.com/trac/ticket/4201 [#2689] twisted.test.test_assertions is lonely (opened by exarkun, closed by jesstess, fixed) defect trial http://twistedmatrix.com/trac/ticket/2689 [#4157] non-local inlineCallbacks exit from returnValue being called in the wrong function is very confusing (opened by glyph, closed by glyph, fixed) enhancement core http://twistedmatrix.com/trac/ticket/4157 [#2116] sshsimpleserver.py on Windows (opened by synapsis, closed by exarkun, worksforme) defect conch http://twistedmatrix.com/trac/ticket/2116 Ticket Lifetime Stats __ Oldest open ticket - [#50] conch command-line client doesn't work in win32 (since 2003-07-12 16:41:06). Newest open ticket - [#4202] Bad markup in doc/web2/howto.object-traversal (since 2010-01-08 15:41:02). Mean open ticket age: 878 days, 16:53:23.432636. Median: 854 days, 5:55:22.572324. Standard deviation: 616 days, 6:48:49.855952. Interquartile range: 941 days, 17:33:57. Mean time between ticket creation and ticket resolution: 211 days, 6:59:00.709972. Median: 27 days, 11:22:47. Standard deviation is 358 days, 16:38:52.703477. The interquartile range is 249 days, 7:15:45. Mean time spent in review: 79 days, 8:39:32.040875. Median: 4 days, 5:07:43. Standard deviation: 273 days, 2:02:52.068017. Interquartile range: 16 days, 20:07:21. Mean number of times a ticket is reviewed: 2.05202312139. Median: 1 Standard deviation: 1.69219379922. Interquartile range: 1. Contributor Stats __ In the last 4 weeks, 20 unique ticket reporters 10 unique ticket reviewers 10 unique ticket resolvers In the last 24 weeks, 103 unique ticket reporters 19 unique ticket reviewers 20 unique ticket resolvers In the last 48 weeks, 179 unique ticket reporters 24 unique ticket reviewers 25 unique ticket resolvers ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python
[Twisted-Python] Lore to Sphinx Conversion Progress Report 4
This time I think I'm gonna skip saying how I haven't gotten as much done as I would like...oh darn. Anyways, time for another gripping installment... Progress: - tables are now handled (mostly) properly, thanks to Zeth at http://commandline.org.uk/ - blockquote tags handled - much improved whitespace/indentation handling - some nicer styling thanks to Michael Thompson - I've managed to convert the docs for the 3 Divmod projects with Lore docs, though I've yet to put them up anywhere. Oustanding issues: - two files in the Lore source are not yet being converted, but it looks like one of them is about to be removed (http://twistedmatrix.com/trac/ticket/4188), and it's not really a Lore doc anyways. - due to ReST's insistence on inline markup being surrounded by whitespace or certain special characters, there are a lot of places where such inline markup gets jacked up, by not including whitespace in front of it. If I put whitespace in front of everything though, my indentation handling gets jacked up and about 400+ Sphinx build warning result. Not sure if I should spend the time to make whitespace handling really smart or if these should just be fixed manually post-conversion. - cite tags still need handling...not hard, just haven't decided the best way to do it yet. - Themeing/styling: still mostly a TODO, though new styling looks a lot better than the default to my eyes. I'm starting to think that eventually we might want to have 2 themes/styles...one to match the trac-based website, and one for bundled docs (docs tarballs, CHM files, etc.) - auto-generated toctree directives are currently generated in alphabetical order, which makes the prev and next links mostly make no sense - some of the Lore source files have nested inline markup, which ReST disallows. This can be handled by: - fix the markup in the Lore source - figure out some kind of supersmart auto-conversion for every possible combination of nesting - just handle the outside level of nesting (what I'm doing now) and fix any problems manually post-conversion. - xhtml entities are not currently resolved...mostly because it makes the build take a LNG time. They can be though. This shouldn't be a problem. - xhtml comments still need to be handled - code class=API tags need something better..right now they are just the same as code tags...Sphinx has an upcoming feature coming in 1.0 that would make this nice and maintainable in the long run, but I don't know that I want to wait for it. I may try to backport the extension or just come up with a separate solution. - some of the generated links need fixing (e.g. links to directories, .py files) In other news: - Foolscap 0.5 was released today, which made me wonder what they use for docs...and it's Lore. I brought this up on IRC, and it was suggested by many that Lore should stick around even after the conversion according to the standard Twisted compatibility policy, to give anyone who still uses it time to migrate. This sounds like a fine idea to me. Any thoughts? As always, the lore2sphinx code is here: http://bitbucket.org/khorn/lore2sphinx/ And the sample output of the conversion process is here: http://twistedsphinx.funsize.net/ Cheers, Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 4
On Thu, Jan 21, 2010 at 3:03 AM, exar...@twistedmatrix.com wrote: On IRC last night I brought up the idea that we could skip the conversion to ReST and use Sphinx with xhtml input documents. The conclusion seemed to be that this might be difficult, but no one was really sure what work would be involved in this approach. Kevin's already put a lot of effort into the conversion. It would be nice if someone else could investigate this. From a docutils point of view, I _think_ it would involve creating a docutisl Parser object, which could read the Lore sources and translate it into a doctree of nodes. From there you could generate the output in whatever format is supported by docutils. This would be doable in theory, but I don't think it would be trivial or even easy, and then you'd have another tool to maintain until the end of time. Sphinx adds a bunch of extensions and conventions on top of docutils proper, and I have little to no idea of what would be involved in handling those issues. - Foolscap 0.5 was released today, which made me wonder what they use for docs...and it's Lore. I brought this up on IRC, and it was suggested by many that Lore should stick around even after the conversion according to the standard Twisted compatibility policy, to give anyone who still uses it time to migrate. This sounds like a fine idea to me. Any thoughts? Since nobody really uses lore's API, the same compatibility policy doesn't really apply. In lore's case, I would say that the policy should be that we include it with X more releases just for packaging convenience, but stop doing maintenance immediately. As long as someone wants to do maintenance, I don't see any reason to stop them from doing it. We might mark all the Lore tickets lowest priority or otherwise signal that some subset of the core developers aren't interested in maintaining it but then, how would that be any different from the status quo? Jean-Paul If someone wants to maintain Lore, I certainly have no objection, but I don't think anyone really does. And it's not like it won't still be in the SVN repos back in the history someplace, even if it were to be removed from trunk. I don't have strong feelings about it either way, but my feeling is Lore should maybe be officially deprecated for a release (or 2, 3, etc.) and then removed from trunk, and if someone wants to maintain it themselves, they can fork it and deal with it outside the Twisted project. Or it can be resurrected later on if someone wants. At the very least, don't force users to install Lore along with the rest of Twisted once Lore is out of general use. Maybe it could be a separate package like web2? But if it were just up to me, I would just get rid of it, to avoid taking up developer time, cluttering up trac with open tickets, etc. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] PyCon 2010 Twisted Sprint
On Sun, Jan 31, 2010 at 4:26 PM, exar...@twistedmatrix.com wrote: Hi all, If there are enough interested people, there will be a Twisted sprint at PyCon this year. If you'll be at PyCon and are interested in sprinting on some Twisted topic, give a shout. Also, let us know what specific topics you're interested in so we can let other people know what to expect in the general announcement. Jean-Paul I plan on being around for most of the first day of sprints (22nd). I'd be interested in most of the topics listed above, but also getting the Generic SASL support working, as well as stdio/console support on Win32. And of course, getting the Twisted Docs transitioned to Sphinx. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Lore to Sphinx Conversion Progress Report 5
This time on How the Docs Conversion Turns... Moving along at about the same pace, but this time have some very visible results! Here's the latest: - The generated toctree directives now preserve the order of the links from the original source documents. This makes the prev/next links work in a sane fashion in the resulting Sphinx docs. - Comments in Lore docs are now translated properly. - You want Trac integration? I got your Trac integration right here! http://twistedsphinx.funsize.net/ OK, so it's not quite done yet, but I was pretty impressed at the results of what amounts to about a day's work. It still needs some tweaks here and there, but I'm mostly satisfied with it. If there are any CSS gurus out there who are willing to take a look at this, there are a few spots that are giving me fits. Send me an email or look me up on IRC. In any case, I'd appreciate it if everyone would take a look and tell me what they think. (Disclaimer: I haven't really tested this in anything but Firefox, except for a brief glance in IE, so if people could test it out in Safari, etc. that would be helpful) - The previous look (which was just the default theme, with a color scheme to match the Twisted website) has been moved into it's own Sphinx theme. Sphinx's theme inheritance made this really easy. This way people can build the docs with the old look if they wish, or maybe we can use it for docs tarballs or .CHM files or something. Still to come: - code class=API tags still need handling. - Certain links need special handling as they don't quite display properly. - Inline markup still needs a bit more work. - XHTML entities need resolving, but as I've mentioned previously, this will probably be the very last thing I address. All in all, I think this is starting to come together. I'm hoping to have the rest completed by PyCon, but if I don't manage that, I'll probably sprint on this on Monday. code is here: http://bitbucket.org/khorn/lore2sphinx/ current converted docs here: http://twistedsphinx.funsize.net/ Until next time.. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 5
On Fri, Feb 5, 2010 at 1:00 PM, Terry Jones te...@jon.es wrote: Hi Kevin I think this is looking really good. Here are a few superficial / quick comments. - It looks fine under Google chrome Good to hear! - You could reduce visual repetition / density of some of the high-level pages by taking out a word like Documentation which appears in the heading and on every line of things like this: Twisted Documentation Table of Contents Twisted Conch Documentation Twisted Core Documentation Historical Documents Twisted Lore Documentation Twisted Mail Documentation Twisted Names Documentation Twisted Pair Documentation Twisted VFS Documentation Twisted Web Documentation Twisted Web2 Documentation Twisted Words Documentation See also http://twistedsphinx.funsize.net/contents.html which is very cluttered with repeated content. I realize these are probably just being picked up out of the .rst header line, and that one of those documents when seen in isolation probably should have a full title. I'm not sure how best to simplify the top-level, though I guess it's possible to extend the sphinx layout.html and override something... may be more trouble than it's worth. A lot of this is because the auto-generated TOC is being shown, and it is indeed picking up the document titles out of the source file it happens to be pointing to. In the final version, I'm thinking we won't really need to show the toctrees at all, since most of the docs have index files already (which I'm generating the toctree directives from), so most of this will fix itself. - I think it would improve appearance a little to have a non-intrusive border around the green source code boxes. Not a bad idea. Or maybe an outline instead of a border? I'll see if I can find/make time to have a look at the CSS. That would be really great. There are a few things that are driving me bonkers. For example: the prev/next/index links on the right hand side of the gray navbar are 1 freakin' pixel lower than the breadcrumb links on the left hand side of the same navbar () at least on this machine), and I have no idea why. Terry ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 5
On Fri, Feb 5, 2010 at 1:11 PM, Terry Jones te...@jon.es wrote: - It looks fine under Google chrome Argh, having said that, I just noticed the table of contents is below and to the right of the main page content. An example page that exhibits this is http://twistedsphinx.funsize.net/projects/conch/howto/conch_client.html I think this has to do with the (extremely hackish, and certainly not very solid) way that the layout is done at the moment. It's a combination of Trac and Sphinx layout, so it ends up kinda goofy, with strange floats and negative margins and whatnot (though not in a way that makes sense to me). I don't see the problem you describe unless I shrink my browser down below a certain minimum size. It could certainly stand improvement. Terry Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 5
On Fri, Feb 5, 2010 at 1:33 PM, Terry Jones te...@jon.es wrote: Hi Kevin I just tried building under Python 2.5.2 and got this: $ python lore2sphinx.py Traceback (most recent call last): File lore2sphinx.py, line 32, in module from table import Table File /home/terry/s/net/lore2sphinx/table.py, line 214 row_lines = list(izip_longest(*row_split, fillvalue='')) ^ SyntaxError: invalid syntax From a quick look at table.py, I don't see the problem. I have to go to a meeting now, so can't dig into it ATM. Also, I have Spinx 0.6.3 installed, not 0.6.4 That's odd...I have no idea what's going on there. I'm actually kind of surprised you even got that far...see below. BTW, why is there a Python 2.6 requirement? I wrote all the FluidDB docs in Sphinx with 2.5 (http://doc.fluidinfo.com/fluidDB/). Terry The original reason for it was because I wanted to use (and am using) os.path.relpath, which was (according to the Python docs) introduced in Python 2.6. It also happeed to be what I had installed on this machine. Since I decided to use 2.6, though, some other 2.6-only stuff might have snuck in, though nothing comes to mind at the moment. If you feel like modifying the code to work on 2.5, please feel free, though since this tool should only be used for real a couple of times, I don't think it's worth spending a whole lot of effort on, unless you just feel like helping out and are stuck on 2.5. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 5
On Fri, Feb 5, 2010 at 2:16 PM, Kevin Horn kevin.h...@gmail.com wrote: On Fri, Feb 5, 2010 at 1:33 PM, Terry Jones te...@jon.es wrote: Hi Kevin I just tried building under Python 2.5.2 and got this: $ python lore2sphinx.py Traceback (most recent call last): File lore2sphinx.py, line 32, in module from table import Table File /home/terry/s/net/lore2sphinx/table.py, line 214 row_lines = list(izip_longest(*row_split, fillvalue='')) ^ SyntaxError: invalid syntax From a quick look at table.py, I don't see the problem. I have to go to a meeting now, so can't dig into it ATM. Also, I have Spinx 0.6.3 installed, not 0.6.4 That's odd...I have no idea what's going on there. I'm actually kind of surprised you even got that far...see below. After looking at this some more, it looks as though izip_longest is the culprit...introduced in 2.6. Why its giving a syntax error, though, is beyond me. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 5
On Fri, Feb 5, 2010 at 2:38 PM, Tristan Seligmann mithra...@mithrandi.netwrote: On Fri, Feb 5, 2010 at 10:18 PM, Kevin Horn kevin.h...@gmail.com wrote: After looking at this some more, it looks as though izip_longest is the culprit...introduced in 2.6. Why its giving a syntax error, though, is beyond me. The problem is the function call syntax: row_lines = list(izip_longest(*row_split, fillvalue='')) ^ SyntaxError: invalid syntax You have keyword arguments after *args, and this is only valid in 2.6 and up. -- mithrandi, i Ainil en-Balandor, a faer Ambar Ah, thanks for that. I don't typically use that syntax, and I thought it looked kind of goofy, but that's the way the signature for izip_longest is in the docs, so I used it there. And since it was introduced in 2.6... All makes sense now... Thanks again! Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 5
On Fri, Feb 5, 2010 at 9:23 PM, Timothy Allen screwt...@froup.com wrote: On Fri, 5 Feb 2010 13:19:00 -0600 Kevin Horn kevin.h...@gmail.com wrote: I'll see if I can find/make time to have a look at the CSS. That would be really great. There are a few things that are driving me bonkers. For example: the prev/next/index links on the right hand side of the gray navbar are 1 freakin' pixel lower than the breadcrumb links on the left hand side of the same navbar () at least on this machine), and I have no idea why. After spending a few minutes poking about with Firebug, the reason becomes clear: - In twistedtrac.css line 38, the navbar list items are set to display: inline, which makes them layout like span and stack up nicely on the left, but also makes them ignore certain vertical adjustments like margin and padding. - In twistedtrac.css line 42, the navbar list items on the right are set to float: right, which makes them stack up nicely on the right but also happens to make them respond to margin and padding again. - In trac.css line 179, the navbar list items are set to padding: 0.25em 0, giving them vertical padding. This is ignored by the inline list items on the left, but respected by the floated list items on the right. Brilliant! It may only have taken you a few minutes, but it would have taken me...considerably longer. :) I haven't studied the CSS of the main Twisted site, so I don't know how they get around it there. On the main twisted site, they don't have anything on the left...not sure how the Sphinx templates handle it though... One easy way around it for the docs might be to change display: inline to float: left, which also makes them stack up on the left, and behave consistently with the ones on the right. Actually that jacks up the height on the navbar completely (at least for me). Overriding the padding in twistedtrac.css seems to work though, so I think that's what I'll go with. Thanks for the help! Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] multiple versions of twisted on same machine
FYI, this totally fails on Windows XP, Py2.6, Twisted 9.0.0... [command line oputput] (venv) C:\Documents and Settings\hornk\Desktop\twisted_venv_test\venveasy_install Twisted Searching for Twisted Reading http://pypi.python.org/simple/Twisted/ Reading http://twistedmatrix.com/ Reading http://www.twistedmatrix.com Reading http://twistedmatrix.com/products/download Reading http://twistedmatrix.com/projects/core/ Best match: Twisted 9.0.0 Downloading http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/Twisted-9.0.0.win32-py2.6.exe Processing Twisted-9.0.0.win32-py2.6.exe error: c:\docume~1\hornk\locals~1\temp\easy_install-ftwcae\Twisted-9.0.0.win32-py2.6.exe is not a valid distutils Windows .exe [end command line output] Am I missing something or should I file a ticket for this? Kevin Horn On Thu, Feb 11, 2010 at 6:55 AM, Matt Bone thatmattb...@gmail.com wrote: You could try virtualenv: http://pypi.python.org/pypi/virtualenv And have one virtual environment with Twisted 9.0 and another with Twisted 8.2, etc, etc. --matt On Thu, Feb 11, 2010 at 6:41 AM, Pet petshm...@googlemail.com wrote: Hi, I'd like to upgrade twisted to the newest version and obviously first test if my application works fine with new version. What is the best (safest) way to do it? Thanks for help, Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] multiple versions of twisted on same machine
On Thu, Feb 11, 2010 at 11:05 AM, exar...@twistedmatrix.com wrote: On 04:41 pm, kevin.h...@gmail.com wrote: FYI, this totally fails on Windows XP, Py2.6, Twisted 9.0.0... This is an interesting problem. easy_install identifies http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/Twisted-9.0.0.win32-py2.6.exe as some kind of package it can install - maybe a self-extracting zip? I don't know - but that's not what it is. This seems to be because the name matches the scheme setuptools uses for naming certain kinds of distribution files. I think it thinks that it's an executable created using python setup.py bdist_wininst or something similar... We presently are not distributing binary eggs for Windows, so easy_install isn't likely to be the best installation strategy anyway. Well, the second thing I do on a new machine (after installing Python of course) is install MinGW and set distutils up to use it as a compiler for Python extensions, so for me it usually works just fine. :) However, in the general case, I agree that most people won't necessarily have a compiler installed on Windows. Not entirely related to this discussion (though not entirely unrelated), I think that Twisted's whole release/distribution system needs to be revamped a little. I've been meaning to talk to radix about this, but haven't found the time...maybe this will give me the kick in the pants I needed... Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] web page on doc...
On Tue, Feb 16, 2010 at 12:17 PM, K. Richard Pixley r...@noir.com wrote: I've been going around and around on documentation with/for twisted for a few days now. For example, I read things like the documentation is written in epytext and documentation is processed by trial and conclude that trial processes epytext. I think I'm beginning to understand what's really going on now but two things would have helped me. 1) a brief overview of the documentation. (proposal below). 2) if references to the documentation could be clarified to refer to howto documentation vs API documentation. If agreed, then I will volunteer to help find and patch #2. As a proposal for #1, I offer the following as an insertion into http://twistedmatrix.com/trac/wiki/ReviewingDocumentation as a second section: UNDERSTANDING DOCUMENTATION ORGANIZATION Documentation for Twisted comes in four primary parts. The first part is a collection of Howto documents which are composed in xhtml, processed by twisted's trial, into html. The source for the Howto documents lives in subversion under doc in the various howto directories. The second part is a collection of examples. These examples are included inline in the howto doc. These are written in python and twisted and are included in the various examples directories under doc in the source. The third part is the API documentation which is produced by pydoctor from the docstrings scattered throughout the twisted source code. The doc strings are written in epytext format. The fourth part are traditional unix man pages. These are written in troff -ms format and the sources are stored in the various man directories in the source under doc. --rich I think that an overview like this is a great idea, though in your text above, the xhtml is processed into html by lore not trial. With any luck this will change soon, and the long-form documentation (how-tos, etc.) will transition to Sphinx. If you'd like to see what that looks like, see here: http://twistedsphinx.funsize.net Other than that, something very like what you have described should definitely be _somewhere_ on the Twisted site. (As an aside, I think a lot of things that are currently on the wiki should be merged into the documentation proper, but that's a topic for another post.) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] web page on doc...
On Tue, Feb 16, 2010 at 2:18 PM, K. Richard Pixley r...@noir.com wrote: Kevin Horn wrote: xhtml is processed into html by lore not trial. Doh. Thanks. I *knew* that. I haven't gotten either to work yet, but neither is really on my radar right now either. Well, I haven't gotten Lore to work (on Windows at least) in a looong time. The only way I can get trial to work on Windows is by using the Twisted Command Prompt, or some hairy PATH manipulation. http://twistedsphinx.funsize.netOther than that, something very like what you have described should definitely be _somewhere_ on the Twisted site. (As an aside, I think a lot of things that are currently on the wiki should be merged into the documentation proper, but that's a topic for another post.) I think I'd need to hear more. I think there's a subtle distinction between describing the code, usage, and interfaces of the twisted library, and describing the people, policies, and interfaces of the twisted project. Generally I think I'd prefer to keep them as clearly segregated as possible but there's certainly some grey area in between. --rich For example, if you want to contribute to Twisted, you need to read (at the very least): http://twistedmatrix.com/documents/current/core/development/index.html but you also need to read: http://twistedmatrix.com/trac/wiki/TwistedDevelopment http://twistedmatrix.com/trac/wiki/ReviewProcess and maybe: http://twistedmatrix.com/trac/wiki/ContributingToTwistedLabs to really get a full picture. A lot of this is incomplete on a given page, and you need to have read the others to really understand what's going on. Some of that stuff should be merged into the actual development docs (like the stuff on commit messages, for example). Some of it certainly belongs in the wiki, but not all of it, and some of that could be condensed into one page. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] [ANNOUNCE] Twisted 10.0.0pre1 is now released
On Sun, Feb 21, 2010 at 9:50 PM, Jonathan Lange j...@mumak.net wrote: Live from PyCon Atlanta, I'm pleased to herald the approaching footsteps of the 10.0 release. Tarballs for the first Twisted 10.0.0 pre-release are now available at: http://people.canonical.com/~jml/Twisted/http://people.canonical.com/%7Ejml/Twisted/ This release is the first release ever with the new NEWS building system, which turns out to be utterly fantastic. We're also using this release to actually hammer out a release process. You can find the draft at: http://twistedmatrix.com/trac/wiki/ReleaseProcess Please feel free to update it with questions, thoughts, corrections and advice. Here's a few...perhaps these should be added to the Open Questions section: 1. How/when in this process are the Windows installers and/or MacOS .dmg files created? (I presume .deb and .rpm packages are left up to Linux distro packagers) 2. How/when in this process are the docs built? 3. When should the front page of the wiki be updated? Thanks, jml No, thank _you_. :) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] [ANNOUNCE] Twisted 10.0.0pre1 is now released
On Mon, Feb 22, 2010 at 12:23 PM, Kevin Horn kevin.h...@gmail.com wrote: On Sun, Feb 21, 2010 at 9:50 PM, Jonathan Lange j...@mumak.net wrote: Live from PyCon Atlanta, I'm pleased to herald the approaching footsteps of the 10.0 release. Tarballs for the first Twisted 10.0.0 pre-release are now available at: http://people.canonical.com/~jml/Twisted/http://people.canonical.com/%7Ejml/Twisted/ This release is the first release ever with the new NEWS building system, which turns out to be utterly fantastic. We're also using this release to actually hammer out a release process. You can find the draft at: http://twistedmatrix.com/trac/wiki/ReleaseProcess Please feel free to update it with questions, thoughts, corrections and advice. Here's a few...perhaps these should be added to the Open Questions section: 1. How/when in this process are the Windows installers and/or MacOS .dmg files created? (I presume .deb and .rpm packages are left up to Linux distro packagers) 2. How/when in this process are the docs built? 3. When should the front page of the wiki be updated? Thanks, jml No, thank _you_. :) Kevin Horn Here's one more: Document how and when to tag releases in SVN. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] PyCon sprint 2010
I was there a bit Sunday night and most of Monday sprinting on the lore-sphinx docs conversion. It's _almost_ done. No really, I mean it! Kevin Horn (khorn on irc) On Thu, Feb 25, 2010 at 3:39 PM, Jonathan Lange j...@mumak.net wrote: Hey guys, There's a bunch of cool stuff on http://twistedmatrix.com/trac/wiki/PyconSprint2010 about what we did at the sprint in PyCon. I want to write this up blog about it for labs.tm.com, but before I do that, I'd like you to look at that page and tell me if anything is missing -- no matter how small. In particular, if you attended and your name isn't there, I want to hear from you. Thanks, jml ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Lore to Sphinx Conversion Progress Report 6
It's been a while, I know you were all waiting with baited breath... First a few fixes: - Thanks to Tim Allen and Steve Steiner, several theme issues were fixed. This fixes a few minor display bugs and should make pages validate properly. - inline markup will now include child contents. This causes rst not to display properly in cases of nested inline markup, but should make it easier to manually fix these places later - :download: links are now generated for non-rst files ending in .py (most of the examples), as welll as .tac and .sql files - toc entries which end in a / character now have index appended to them in the ReST output, which fixes a few issues where the built-Sphinx docs would end up with broken navigation links. A couple of issues were also fixed at the PyCon sprints: - cite tags are now handled...sort of. According to Glyph, these aren't really very important, so we just return the contents of the tag without any special markup. If we feel the need to put citation markup back into the docs we can do that post-transition. - Glyph also found an easy way to improve the CSS of the theme somewhat. The issue where the sidebar drops below the main content should now be less of a problem, though it's not gone entirely. - Glyph and I also discussed a few different ways to handle links to the API docs and agreed on a way forward. Basically, well change the code class='API' tags into something like: :api:`path.to.documented object label`, and then create a docutils extension to convert that into a link. The beginnings of this are in the hg repos, but it looks like it won't be quite as simple as we thought. First, the way pydoctor generates links is deceptively simple. If you look up a module, like twisted.internet.defer, the link looks like this: http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.html If you look up a class, like twisted.internet.defer.Deferred for example, the link looks like this: http://twistedmatrix.com/documents/current/api/twisted.internet.defer.Deferred.html Pretty simple right? But if you look up a function or method like twisted.internet.defer.Deferred.callback, the link has an anchor, like so: http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.Deferred.html#addCallback So we'll need to do some checking on either the api docs or on twisted itself to determine what sort of object is being linked to. Shouldn't be super-difficult, just haven't had time yet. Second, I haven't yet figured out how to access the Sphinx config object from a docutils role function. I'm sure this is just a matter of finding an example or getting an answer from the mailing list, but I just haven't gotten to it yet. I think the API links are the only major issue still remaining until we start doing the conversion, so once this issue is handled, I'll probably file a ticket and create a branch in SVN for the conversion. If anyone has any docs tickets open, now is the time to get them fixed! And if anyone has some great ideas about how to organize the new docs in the repository, I'd love to hear them. As always, converted docs are here: http://twistedsphinx.funsize.net/ The lore2Sphinx Hg repos is here: http://bitbucket.org/khorn/lore2sphinx/ Comments, ideas, patches and pull requests gratefully accepted (or at least discussed). Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Thu, Mar 4, 2010 at 5:05 PM, exar...@twistedmatrix.com wrote: On 10:49 pm, kevin.h...@gmail.com wrote: If you look up a class, like twisted.internet.defer.Deferred for example, the link looks like this: http://twistedmatrix.com/documents/current/api/twisted.internet.defer.Deferred.html Pretty simple right? But if you look up a function or method like twisted.internet.defer.Deferred.callback, the link has an anchor, like so: http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.Deferred.html#addCallback So we'll need to do some checking on either the api docs or on twisted itself to determine what sort of object is being linked to. Shouldn't be super-difficult, just haven't had time yet. Did you notice that http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.Deferred.addCallback.html also exists? Jean-Paul I hadn't actually. If all functions and methods have a similar page in the PyDoctor output, then I think what I have now will pretty much work as-is (almost). Do we think that just generating links to the individual function/method pages is acceptable? Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Thu, Mar 4, 2010 at 6:10 PM, Michael Hudson-Doyle mica...@gmail.comwrote: On 5 March 2010 12:31, Kevin Horn kevin.h...@gmail.com wrote: On Thu, Mar 4, 2010 at 5:05 PM, exar...@twistedmatrix.com wrote: On 10:49 pm, kevin.h...@gmail.com wrote: If you look up a class, like twisted.internet.defer.Deferred for example, the link looks like this: http://twistedmatrix.com/documents/current/api/twisted.internet.defer.Deferred.html Pretty simple right? But if you look up a function or method like twisted.internet.defer.Deferred.callback, the link has an anchor, like so: http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.Deferred.html#addCallback So we'll need to do some checking on either the api docs or on twisted itself to determine what sort of object is being linked to. Shouldn't be super-difficult, just haven't had time yet. Did you notice that http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.Deferred.addCallback.html also exists? Jean-Paul I hadn't actually. If all functions and methods have a similar page in the PyDoctor output, then I think what I have now will pretty much work as-is (almost). Do we think that just generating links to the individual function/method pages is acceptable? It's what the lore-processed documentation currently links to. It would be nice to fix this (and I sort of vaguely know how), but that should in no way hold up the move away from lore! Cheers, mwh OK, if that's the way Lore does it, then I'm happy to call that good enough for now. It seems to work here in my local version of the docs, I'll update the online stuff at http://twistedsphinx.funsize.net soonish. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Fri, Mar 5, 2010 at 9:30 AM, Drew Smathers drew.smath...@gmail.comwrote: On Thu, Mar 4, 2010 at 5:49 PM, Kevin Horn kevin.h...@gmail.com wrote: It's been a while, I know you were all waiting with baited breath... First a few fixes: - Thanks to Tim Allen and Steve Steiner, several theme issues were fixed. This fixes a few minor display bugs and should make pages validate properly. - inline markup will now include child contents. This causes rst not to display properly in cases of nested inline markup, but should make it easier to manually fix these places later - :download: links are now generated for non-rst files ending in .py (most of the examples), as welll as .tac and .sql files - toc entries which end in a / character now have index appended to them in the ReST output, which fixes a few issues where the built-Sphinx docs would end up with broken navigation links. A couple of issues were also fixed at the PyCon sprints: - cite tags are now handled...sort of. According to Glyph, these aren't really very important, so we just return the contents of the tag without any special markup. If we feel the need to put citation markup back into the docs we can do that post-transition. - Glyph also found an easy way to improve the CSS of the theme somewhat. The issue where the sidebar drops below the main content should now be less of a problem, though it's not gone entirely. - Glyph and I also discussed a few different ways to handle links to the API docs and agreed on a way forward. Basically, well change the code class='API' tags into something like: :api:`path.to.documented object label`, and then create a docutils extension to convert that into a link. The beginnings of this are in the hg repos, but it looks like it won't be quite as simple as we thought. First, the way pydoctor generates links is deceptively simple. If you look up a module, like twisted.internet.defer, the link looks like this: http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.html If you look up a class, like twisted.internet.defer.Deferred for example, the link looks like this: http://twistedmatrix.com/documents/current/api/twisted.internet.defer.Deferred.html Pretty simple right? But if you look up a function or method like twisted.internet.defer.Deferred.callback, the link has an anchor, like so: http://twistedmatrix.com/documents/9.0.0/api/twisted.internet.defer.Deferred.html#addCallback So we'll need to do some checking on either the api docs or on twisted itself to determine what sort of object is being linked to. Shouldn't be super-difficult, just haven't had time yet. Second, I haven't yet figured out how to access the Sphinx config object from a docutils role function. I'm sure this is just a matter of finding an example or getting an answer from the mailing list, but I just haven't gotten to it yet. I think the API links are the only major issue still remaining until we start doing the conversion, so once this issue is handled, I'll probably file a ticket and create a branch in SVN for the conversion. If anyone has any docs tickets open, now is the time to get them fixed! How can I filter for tickets related to lore2sphinx transition in trac? I've noticed a few documents that don't seem properly converted. Here's an example: http://twistedsphinx.funsize.net/projects/web/howto/web-in-60/asynchronous.html ... which is missing syntax highlighting on code examples and the code examples are also truncated. -Drew As far as I know, there aren't any tickets in Trac yet. I was planning on making some (or asking others to do so) once a master ticket and a branch are created. As for the specific issue you linked to, the ReST markup is similar to that on other pages, but it looks like Sphinx isn't recognizing the code samples as Python for some reason. I'll look into this later on. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] http://twistedmatrix.com/ trac installation updated
On Thu, Mar 11, 2010 at 2:05 PM, exar...@twistedmatrix.com wrote: Hi all, Thomas and I have finished upgrading the trac installation on http://twistedmatrix.com/ to 0.11.6. We expect that this will resolve a number of the issues the site has had. If you notice anything broken about the website now, please point it out. Jean-Paul Here's a minor thing... When viewing a diff, the legend is incomplete. I just see text (Unmodified, Added, etc.) without any images/colors. It's a tiny thing, but you did ask... :) browser: Firefox 3.5 on Win32 Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] cron-style scheduler for Twisted
On Wed, Apr 14, 2010 at 8:11 AM, vit...@synapticvision.com wrote: hi, is there any cron-style scheduler in Twisted? Could one point me to the docs where it could be found? Thanks. I wrote something like this once, I called it txScheduler. I guess I really should clean it up and release it. It's not been heavily tested (and I never got around to writing the unit tests I had intended to) but it worked for the original use case I had. If anyone's interested, drop me a line directly and I'll send it to you (MIT license). Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted.web in 60 seconds...why aren't these on the frontpage of TM.com?
This blog series is also totally rock-a-licious. http://krondo.com/blog/?page_id=1327 Kevin Horn On Wed, Apr 21, 2010 at 8:22 PM, Jason J. W. Williams jasonjwwilli...@gmail.com wrote: Naw...I meant what I wrote. ;) exarkun's is great...it's web focused. I'll post to them to the list and y'all can see if its worthwhile. -J On Wed, Apr 21, 2010 at 7:05 PM, sstein...@gmail.com sstein...@gmail.com wrote: On Apr 21, 2010, at 8:53 PM, Jason J. W. Williams wrote: But because I'm pretty comfortable with Twisted now...it makes more sense for me to write my own dispatcher that emulates Tornado's than give up all the rock-hard awesome My spam filter almost killed you on that phrase. I think you mean rock-solid. ...so my point is...Twisted's PR problem is it needs more accessible docs. They need to be understandable, entertaining...and pretty. It all starts with that front page...and the web app in 10 lines right smack dab there. I'm going to start with the front page...and then I'm going to write some articles on my blog that are what I wanted when I was learning Twisted. Hopefully, they'll be of use to someone. Cool. See exarkun's Twisted in 60 Seconds series; maybe fill in the things that are still not clear after that? S ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted.web in 60 seconds...why aren't these on the frontpage of TM.com?
On Wed, Apr 21, 2010 at 7:53 PM, Jason J. W. Williams jasonjwwilli...@gmail.com wrote: I'd say the biggest barrier to entry is the documentation. And let's face it...there's no reason Twisted shouldn't be used by more folks. It's great...but learning it feels like a frat hazing. That's largely because reading the docs feels like learning from that comp sci prof who was smart...but wasn't smart enough to explain the concepts at your level. The docs really are about to get a whole lot better. No really, any minute now. The Sphinx conversion is not dead! I've just been busy... And once that's done I think it'll be a whole lot easier to write and maintain things, not to mention read the docs. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twistd and init.d
Thanks for reporting back to the list, Don. I'm sure this info will help
someone down the road (maybe even me!)
Kevin Horn
On Thu, Apr 22, 2010 at 9:11 AM, Don Schoeman d...@delphexonline.com wrote:
Hi guys,
I have finally managed to solve this problem. It turns out (as predicted)
that this is not a twistd problem after all. This particular Ubuntu server
has an encrypted home directory (enabled through eCryptfs). This means that
during the boot process the particular home directory from which I tried
running my tests was not available as even the root user do not have access
to the home directory until it is de-crypted after the particular user
actually logs in. This is why the tests ran fine after login but not during
boot.
For more experienced Linux admins this would have been tracked down within
minutes but it took me a full day worth of testing before I finally managed
to track down the problem. To resolve the issue I moved the server to
/usr/local/lib/python2.6/dist-packages/ghserver and I am now executing it
from there instead and it starts up during boot just as expected. I am
extremely relieved!
Thanks all again for your help!
Best Regards,
Don
Don Schoeman wrote:
Hi Jason,
Unfortunately I still have the same problem. However, I now realise that it
might be a privileges issue of some kind. Being fairly new at linux this
may take days to figure out; can you perhaps remember if you enabled your rc
script using the command update-rc.d? I used it in the following way: sudo
update-rc.d ghserver defaults
Since this doesn't seem to be a problem related to twisted or twistd
anymore I will start looking elsewhere for possible solutions.
Thanks for all the help from everyone anyway.
Regards,
Don
Jason J. W. Williams wrote:
Hi Don,
Here's the init.d we use for Twisted on Ubuntu. Sets the full path to Twistd:
http://gist.github.com/373978
-J
On Wed, Apr 21, 2010 at 5:38 AM, Don Schoeman d...@delphexonline.com
d...@delphexonline.com wrote:
Hi guys,
My application is being executed using twistd as per documentation. All is
working fine but I need to have the service start automatically at boot
time.
So as a test I've created a script in /etc/init.d/ which looks like this:
#! /bin/sh
### BEGIN INIT INFO
# Provides: ghserver
# Required-Start:$local_fs $remote_fs $network $syslog
# Required-Stop: $local_fs $remote_fs $network $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start/stop GHServer server
### END INIT INFO
logger GHServer: Start script executed
GH_SERVER_PATH=/home/myname/Python/ghserver
export PYTHONPATH=$GH_SERVER_PATH:$PYTHONPATH
case $1 in
start)
logger GHServer: Starting
echo Starting GHServer...
twistd -y $GH_SERVER_PATH/ghserverapp.py -l
$GH_SERVER_PATH/ghserver.log --pidfile $GH_SERVER_PATH/twistd.pid
;;
stop)
logger GHServer: Stopping
echo Stopping GHServer...
kill `cat $GH_SERVER_PATH/twistd.pid`
;;
*)
logger GHServer: Invalid usage
echo Usage: /etc/init.d/ghserver {start|stop}
exit 1
;;
esac
exit 0
This works fine when running /etc/init.d/ghserver start and
/etc/init.d/ghserver stop. The script also run when I boot since the logger
actually logs the GHServer: Starting text to the /var/log/messages file.
However, my service actually does not start. There is no pid to be found
anywhere, there are no error logs anywhere, just nothing. I might be doing
something wrong here but is there someone who's gone through this process
and can provide some samples how they did it? Any help will be greatly
appreciated.
Kind Regards,
Don
___
Twisted-Python mailing
listtwisted-pyt...@twistedmatrix.comhttp://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing
listtwisted-pyt...@twistedmatrix.comhttp://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
--
___
Twisted-Python mailing
listtwisted-pyt...@twistedmatrix.comhttp://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted.web in 60 seconds...why aren't these on the frontpage of TM.com?
On Wed, Apr 21, 2010 at 9:48 PM, exar...@twistedmatrix.com wrote: So about the conversion, I wonder if we can start to generate the Sphinx version of the docs on buildbot sometime soon, and perhaps host them alongside the current docs, as the first step in phasing out the current version? I suppose we could do this, if you think it would help. Note that the lore2sphinx script requires Python 2.6 and I think at least Sphinx 0.6.4 (I'm using 0.6.5). How automated is the conversion process now? Are errors reported such that people could start looking at them and fixing the issues? It seems like you've done almost all of the hard work, and the rest is dealing with some edge cases in lore or restructured text, and that might be more easily spread out amongst more people. Jean-Paul I think the automatic conversion process has reached the point of diminishing returns. I've got it down to about 13 warnings in the Sphinx build process, though there are still some markup errors in the output. Trying to make the conversion handle these seems to just introduce more errors in other places. In other words I think it's time to create a branch with the Sphinxified docs and do a manual review. I've got a basic transition plan sketched out here: http://twistedsphinx.funsize.net/transition_plan.html That's just basically an outline of a plan Glyph and I hashed out during the Twisted sprint at PyCon. Ideas and criticisms welcome. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted.web in 60 seconds...why aren't these on the frontpage of TM.com?
On Thu, Apr 22, 2010 at 11:51 AM, exar...@twistedmatrix.com wrote: On 02:57 pm, kevin.h...@gmail.com wrote: On Wed, Apr 21, 2010 at 9:48 PM, exar...@twistedmatrix.com wrote: So about the conversion, I wonder if we can start to generate the Sphinx version of the docs on buildbot sometime soon, and perhaps host them alongside the current docs, as the first step in phasing out the current version? I suppose we could do this, if you think it would help. Note that the lore2sphinx script requires Python 2.6 and I think at least Sphinx 0.6.4 (I'm using 0.6.5). Shouldn't be a problem. How automated is the conversion process now? Are errors reported such that people could start looking at them and fixing the issues? It seems like you've done almost all of the hard work, and the rest is dealing with some edge cases in lore or restructured text, and that might be more easily spread out amongst more people. Jean-Paul I think the automatic conversion process has reached the point of diminishing returns. I've got it down to about 13 warnings in the Sphinx build process, though there are still some markup errors in the output. Trying to make the conversion handle these seems to just introduce more errors in other places. Would it be easier to fix these by changing the lore sources? Jean-Paul I seriously doubt it. In fact I think it would be much more difficult. Other than the 13 build warnings I mentioned above, almost all of the rest of the markup errors are things like a missing space character. Fixing this in the lore sources would require: 1) removing most if not all indentation from all the lore sources 2) making sure every b, i, code, a tag (and some others I'm sure) have exactly the same spacing before and after them...including tabs, newlines, etc. 3) revamping lore2sphinx to stop doing all the workarounds it currently does I think it will be much easier to go through the ReST sources (which IMO will need to be done anyway) and add spaces or whatever where needed. Kevin Horn Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Dropping Python 2.4 support on Windows after 10.1
On Tue, Jun 8, 2010 at 11:20 PM, exar...@twistedmatrix.com wrote: Hey all, I would like to propose that after Twisted 10.1 is released, we cease to claim support for Python 2.4 on any version of Windows. This seems reasonable to me considering that Python 2.5 was first released in September of 2006 and that either Windows Python installations can be upgraded easily or very likely no upgrades are possible at all (specifically, no upgrades of Twisted). The motivation for this is that we do not have the hardware resources to continue to test Twisted on Python 2.4 on Windows, particularly considering we will shortly want to begin testing on Python 2.7 on Windows. If Windows/Python 2.4 support is important enough to anyone that they can offer to host the build slaves necessary to continue testing, then I'll withdraw the suggestion. Lastly, I am not proposing dropping Python 2.4 support on any other platform (ie, Debian and Fedora). Jean-Paul It would be nice to keep it if possible, but if it's a choice between a buildslave for 2.4 and one for 2.7, then 2.7 really needs to take priority. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] making Screwtape (Tim Allen) a committer
+1 On Wed, Jun 9, 2010 at 5:25 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: I think we should give screwtape an SVN account and make him an official reviewer. He's been doing a lot of work on the distutils code which would be made easier by that, and he's been doing pretty decent reviews. Thoughts? (Feel free to voice any comments off-list, too.) ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Multiple clients
On Fri, Jun 18, 2010 at 12:55 PM, Szabolcs Balogh bszabo...@gmail.comwrote: It is possible to: - start the reactor when I initialize my program (eg. in main) - call connectTCP or connectSSL when I want and every time I want? Yes, this is exactly what you are supposed to do. Call the reactor _once_ and once only. The call connectTCP or whatever whenever is appropriate for your program. Something like this: def check_messages(account): # connect to imap server and check for new mailboxes/messages reactor.connectSSL(...) if __name__ == __main__: reactor.run() check_message(account1) check_message(account2) ... do something else ... check_message(accountn) Because I can't implement this :} I have started the implementation based on Twisted Network Programming Essential, but this book doesn't treat multiple connections... The above won't work, because run() blocks. So in your example here, check_messages() won't be called until after the run() call returns, which means the reactor won't be running, which means check_messages won't work. look into callWhenRunning: http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.interfaces.IReactorCore.html#callWhenRunning and do something like: def start_checking(): (do your check_messages stuff here) if __name__ == '__main__': reactor.callWhenRunning(start_checking) reactor.run() using callWhenRunning isn't necessary, but I find that it clarifies things for me sometimes reactor.run() is usually the _last_ thing I call in my Twisted programs, unless I have some shutdown cleanup to do, and even that is usually better done in the reactor shutdown hook (see: http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.interfaces.IReactorCore.html#addSystemEventTrigger ) you also need to figure out how to structure your program flow using deferreds, which is not always easy, and depends greatly on your specific application. remember that deferreds are just a way of organizing callback functions you can chain deferreds something like this (assuming your functions return Deferreds): d = check_first_one() d.addCallback(check_second_one, ...) etc... which will make check_second_one start after check_first_one completes You can also use a deferredList, which is often handy when you want to keep track of several deferred operations which are operating simultaneously (I haven't used one in a while, so I don't have a simple example off the top of my head, but there's a simple example in the Twisted book) Hope this helps, Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Review process, news fragments
+1 On Sat, Jun 19, 2010 at 8:08 AM, exar...@twistedmatrix.com wrote: Hello Twisted developers, Please take a few minutes to refresh your memory of the contents of http://twistedmatrix.com/trac/wiki/ReviewProcess. In particular, I'd like to draw everyone's attention to the requirements for news fragments. Since we introduced this part of the workflow, the review requirements for these has been a little unclear. Many reviewers haven't required that the news fragment be reviewed (that is, if it was missing, they would say Please add a news fragment and then merge.) or haven't kept the guidelines these files in mind when reviewing them. I'd like for this to change. Informative, consistently written news fragments result better information being available to users when we do a release. This is a very simple part the development process but effort here has a disproportionately large effect on the perception of Twisted. If the guidelines for news fragment content on the review process wiki page are unclear or insufficient, please complain and we can work on improving the weaknesses. Thanks, and congratulations to everyone on the (now inevitable :) 10.1 release. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted 10.1.0pre1 released -- please test!
Running trial with the --unclean-warnings flag, I get only 3 failures.
Attached.
Kevin Horn
On Fri, Jun 25, 2010 at 10:46 AM, Kevin Horn kevin.h...@gmail.com wrote:
I pulled down the tarball and installed using 'setup.py install' and ran
the tests on Win XP + Py2.6
A number of tests failed, though I'm not sure how many are really
meaningful. Mostly seems to be
'reactor unclean' type stuff that doesn't happen on every run.
At any rate, output of test failures attached. Hope it's useful.
Kevin Horn
On Fri, Jun 25, 2010 at 7:59 AM, Jonathan Lange j...@mumak.net wrote:
On Fri, Jun 25, 2010 at 12:57 AM, Glyph Lefkowitz
gl...@twistedmatrix.com wrote:
...
Please download the tarballs and test them as much as possible.
...
In particular, you should note that if Twisted 10.1.0 pre1 horribly
breaks
your software, you can pipe up and say hey! don't release it yet!,
and we
may listen to you.
...
On the other hand, if you don't test any prereleases, and you discover
that
your application is broken by Twisted 10.1 the day after the *final*
release, you will be screwed.
...
If you have software that depends on Twisted, it is definitely in your
best
interest to test the pre-releases and let us know how it worked out.
...
I plan to do the final release this weekend, unless I hear any objections.
jml
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
===
[ERROR]: twisted.test.test_plugin.DeveloperSetupTests.test_developmentPluginAvai
lability
Traceback (most recent call last):
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\test\test_plugin.py, line 385, in setUp
True, 'plugindummy_builtin')
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\test\test_plugin.py, line 357, in _createPluginDummy
plugs.child(pluginModule + '.py').setContent(pluginContent)
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\python\filepath.py, line 745, in setContent
f = sib.open('w')
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\python\filepath.py, line 495, in open
return self.create()
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\python\filepath.py, line 771, in create
fdint = os.open(self.path, _CREATE_FLAGS)
exceptions.OSError: [Errno 2] No such file or directory: 'C:\\Documents and Sett
ings\\funsize\\Desktop\\Twisted-10.1.0pre1.tar\\Twisted-10.1.0pre1\\_trial_temp\
\twisted.test.test_plugin\\DeveloperSetupTests\\test_developmentPluginAvailabili
\\43lvnu\\temp\\system_path\\plugindummy\\plugins\\b1J3lmxYhIK04qjOplugindummy_b
uiltin.py.new'
===
[ERROR]: twisted.test.test_plugin.DeveloperSetupTests.test_freshPyReplacesStaleP
yc
Traceback (most recent call last):
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\test\test_plugin.py, line 388, in setUp
True, 'plugindummy_builtin')
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\test\test_plugin.py, line 357, in _createPluginDummy
plugs.child(pluginModule + '.py').setContent(pluginContent)
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\python\filepath.py, line 745, in setContent
f = sib.open('w')
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\python\filepath.py, line 495, in open
return self.create()
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\python\filepath.py, line 771, in create
fdint = os.open(self.path, _CREATE_FLAGS)
exceptions.OSError: [Errno 2] No such file or directory: 'C:\\Documents and Sett
ings\\funsize\\Desktop\\Twisted-10.1.0pre1.tar\\Twisted-10.1.0pre1\\_trial_temp\
\twisted.test.test_plugin\\DeveloperSetupTests\\test_freshPyReplacesStalePyc\\za
dri7\\temp\\development_path\\plugindummy\\plugins\\CZIjtvOP6Kqa-6npplugindummy_
builtin.py.new'
===
[ERROR]: twisted.test.test_plugin.DeveloperSetupTests.test_newPluginsOnReadOnlyP
ath
Traceback (most recent call last):
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\test\test_plugin.py, line 388, in setUp
True, 'plugindummy_builtin')
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\test\test_plugin.py, line 357, in _createPluginDummy
plugs.child(pluginModule + '.py').setContent(pluginContent)
File C:\Python26\lib\site-packages\twisted-10.1.0pre1-py2.6-win32.egg\twisted
\python\filepath.py, line 745, in setContent
f = sib.open('w')
File C:\Python26\lib\site-packages\twisted
Re: [Twisted-Python] Twisted 10.1.0pre1 released -- please test!
OK, running the tests in a higher directory (to shorten temp file path lengths), and with the --unclean-warnings flag, I'm down to 1 failure, which still looks like it could be some kind of transient issue, but I'm not sure. Results attached. Kevin Horn On Fri, Jun 25, 2010 at 11:28 AM, Kevin Horn kevin.h...@gmail.com wrote: Running trial with the --unclean-warnings flag, I get only 3 failures. Attached. Kevin Horn On Fri, Jun 25, 2010 at 10:46 AM, Kevin Horn kevin.h...@gmail.com wrote: I pulled down the tarball and installed using 'setup.py install' and ran the tests on Win XP + Py2.6 A number of tests failed, though I'm not sure how many are really meaningful. Mostly seems to be 'reactor unclean' type stuff that doesn't happen on every run. At any rate, output of test failures attached. Hope it's useful. Kevin Horn On Fri, Jun 25, 2010 at 7:59 AM, Jonathan Lange j...@mumak.net wrote: On Fri, Jun 25, 2010 at 12:57 AM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: ... Please download the tarballs and test them as much as possible. ... In particular, you should note that if Twisted 10.1.0 pre1 horribly breaks your software, you can pipe up and say hey! don't release it yet!, and we may listen to you. ... On the other hand, if you don't test any prereleases, and you discover that your application is broken by Twisted 10.1 the day after the *final* release, you will be screwed. ... If you have software that depends on Twisted, it is definitely in your best interest to test the pre-releases and let us know how it worked out. ... I plan to do the final release this weekend, unless I hear any objections. jml ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python [ERROR]: twisted.web.test.test_cgi.CGI.testReadInput Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: twisted.web.test.test_cgi.CGI tes tMethod=testReadInput (testReadInput) still running at 5.0 secs --- Ran 5946 tests in 443.016s FAILED (skips=831, expectedFailures=12, errors=1, successes=5100)___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Tue, Jul 6, 2010 at 1:27 PM, Jonathan Lange j...@mumak.net wrote: On Tue, Jul 6, 2010 at 8:16 PM, Jonathan Lange j...@mumak.net wrote: On Fri, Mar 5, 2010 at 7:33 PM, Kevin Horn kevin.h...@gmail.com wrote: ... As far as I know, there aren't any tickets in Trac yet. I was planning on making some (or asking others to do so) once a master ticket and a branch are created. I have filed such a master ticket here: http://twistedmatrix.com/trac/ticket/4553 And created a milestone here: http://twistedmatrix.com/trac/milestone/Lore%20to%20Sphinx You should probably add any new tickets you file to the milestone. Hey and look a ticket was already filed (#4500) and http://twistedsphinx.funsize.net/transition_plan.html is out of date :) jml Doh! Sorry about that... I didn't realize anyone was actually looking at it. :) So now that Jonathan has cut the 10.1 release, my plan is to start the conversion process pretty soon (meaning sometime between tonight and Wednesday). Once I have the branch created and the basic conversion completed it would be really great to have some help with the (yet to be created) chunk tickets. We're getting really close... Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Fri, Jul 9, 2010 at 9:58 PM, exar...@twistedmatrix.com wrote: On 6 Jul, 08:09 pm, kevin.h...@gmail.com wrote: So now that Jonathan has cut the 10.1 release, my plan is to start the conversion process pretty soon (meaning sometime between tonight and Wednesday). Once I have the branch created and the basic conversion completed it would be really great to have some help with the (yet to be created) chunk tickets. We're getting really close... At last I've got a buildbot set up generating the sphinx docs. The build results can be seen here: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/ (or with a different revision number for different revisions; the containing directory is browseable). cool, good work. It would be great if everyone could go look around and report any problems you see. Here's one from me. On http://buildbot.twistedmatrix.com/builds /sphinx-html-15615/projects/core/howto/pb-intro.html, there are a bunch of API links which don't appear to have been rendered properly. Ah, welcome to the joys of nested inline markup, which xml/xhtml does fine and RestructuredText doesn't. I expect this issue and extra or missing space characters are the two main things which will have to be fixed manually. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Fri, Jul 9, 2010 at 10:52 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Jul 9, 2010, at 10:58 PM, exar...@twistedmatrix.com wrote: At last I've got a buildbot set up generating the sphinx docs. The build results can be seen here: Thanks for setting this up. In the table of contents, both the Core and Conch outlines have a section called Twisted Documentation. I assume this is a placeholder for a directory that Lore didn't have a title for? Nope, that's what the titles of those documents are in the source documents. I presume the Conch one is supposed to say something else. It strikes me as somewhat ironic that the table on http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/lore/howto/lore.html came out rather nicely, though. Yes, I spent probably too much time on that table, but it sure was handy. I spent a lot of time during this process referring to it, and it was much nicer on the eyes that the old one :). I had originally intended to just convert the tables by hand (I think there's only two), but I found some code on the net that did almost what I wanted and adapted it, so it actually wasn't too onerous. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
to be fixed manually. Some extra thoughts: - The ReviewingDocumentation wiki page has a section called Editing man pages that describes how to turn the nicely-formatted manpages into Lore input files. Would it be possible to do that as part of the lore2sphinx run, have the manpages included in the Sphinx documentation, and from then on generate the manpages from the .rst files instead of the other way around? Sphinx does have a man page builder now, but I don't think it existed when I was writing lore2sphinx, so I haven't really considered this. So you're suggesting convert the man pages to Lore format - use lore2sphinx to convert the resulting Lore docs to rst - build as part of the Sphinx process, yes? I think this is a worthwhile idea, but I'd prefer to leave it until after the main docs are converted (i.e. under a separate ticket). lore2sphinx can be used on just the man files later on if need be, though it would take a little mucking around. [1]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ conch/howto/conch_client.html [2]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ conch/examples/index.html [3]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ core/upgrades/2.0/split.html [4]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ core/upgrades/2.0/components.html [5]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ core/specifications/banana.html [6]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ core/development/policy/coding-standard.html [7]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ core/development/policy/doc-standard.html [8]: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/ core/development/policy/svn-dev.html Thanks for the fantastic (and nicely detailed) feedback, Tim! Please take a look at the transition plan. In a few days (maybe sooner), I should have the base docs in a branch, and the chunk tickets referenced in the transition plan created. This is pretty much _exactly_ what I'd like to see in those chunk tickets. Hopefully you haven't already burned up your brain staring at markup issues. :) We could really use this kind of help throughout the process. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Sat, Jul 10, 2010 at 8:19 PM, Tim Allen screwt...@froup.com wrote: On Sat, Jul 10, 2010 at 03:17:03PM -0500, Kevin Horn wrote: On Sat, Jul 10, 2010 at 2:40 AM, Tim Allen screwt...@froup.com wrote: That's probably enough feedback to be getting on with; most of the problems appear to be from normalising \n in Lore docs to instead of , and also from adding whitespace after things. It is generally looking pretty great, though! Yeah, that's pretty much it. As I said above though, if you fix it one place, it breaks in another, so I tried to balance things in such a way that the least broken markup appears in the output. Almost all of the remaining problems will need to be fixed manually. It sounds like a lot of the manual fixes will involve rewriting things so that inline markup does not appear at the end of a sentence next to the punctuation. That seems like a recipe for awkward prose. :/ Not necessarily. There are ways around a lot of these things, but they make the markup a bit more complex and thus more complicated to generate automatically. For example, if the markup requires that there be a space, but you don't actually want a space there, you insert an 'escaped space' (\ , that's a backslash-space). But the exact rules for what can abut the beginning and end of inline markup are more complicated than that, and accounting for every special case would beer... Let's just say there's a reason that there's not an official docutils tool to _create_ restructuredText. Some extra thoughts: - The ReviewingDocumentation wiki page has a section called Editing man pages that describes how to turn the nicely-formatted manpages into Lore input files. Would it be possible to do that as part of the lore2sphinx run, have the manpages included in the Sphinx documentation, and from then on generate the manpages from the .rst files instead of the other way around? Sphinx does have a man page builder now, but I don't think it existed when I was writing lore2sphinx, so I haven't really considered this. So you're suggesting convert the man pages to Lore format - use lore2sphinx to convert the resulting Lore docs to rst - build as part of the Sphinx process, yes? Yes. At least for trial, there's a bunch of stuff that's *only* in the man page and not the online docs, and a bunch of stuff that's *only* in the online docs and not the man page (and stuff that's *only* in the core/development/policy section of the docs, and not in the Trial section...). Hopefully if they're all part of the same doc system, it'll be easier to have everything in the one place and easy to find. I think this is a worthwhile idea, but I'd prefer to leave it until after the main docs are converted (i.e. under a separate ticket). lore2sphinx can be used on just the man files later on if need be, though it would take a little mucking around. Something to add to the open ticket for: anything else??? section of your transition plan, then? :) Sounds like it. Thanks for the fantastic (and nicely detailed) feedback, Tim! Please take a look at the transition plan. In a few days (maybe sooner), I should have the base docs in a branch, and the chunk tickets referenced in the transition plan created. This is pretty much _exactly_ what I'd like to see in those chunk tickets. Having done pretty much the first two of your suggested chunks in my previous mail, they look to be about the right size. Hopefully you haven't already burned up your brain staring at markup issues. :) We could really use this kind of help throughout the process. I'm looking forward to learning a little more about Sphinx and ReST. :) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Lore to Sphinx Conversion Progress Report 6
On Sat, Jul 10, 2010 at 9:21 PM, Glyph Lefkowitz gl...@twistedmatrix.comwrote: On Jul 10, 2010, at 4:17 PM, Kevin Horn wrote: On Sat, Jul 10, 2010 at 2:40 AM, Tim Allen screwt...@froup.com wrote: On Sat, Jul 10, 2010 at 02:58:40AM -, exar...@twistedmatrix.comwrote: At last I've got a buildbot set up generating the sphinx docs. The build results can be seen here: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/ (or with a different revision number for different revisions; the containing directory is browseable). It would be great if everyone could go look around and report any problems you see. - In the Documentation Table Of Contents page, Historical Documents is listed between Twisted Core and Twisted Lore (presumably because of alphabetical order) but this seems a strange place for it. Could it be moved to the bottom, or an appendix, or something? They're in alphabetical order. The TOC page is generated dynamically by looking at the file structure of the docs and that's the order file-globbing lists them in. We can fix this manually post-conversion. It's worth noting that even in this peculiar order, the existence of this document is an upgrade. It did not previously exist. Or, rather, it did, but I think the analogous page in the current system is this: http://twistedmatrix.com/documents/10.1.0/, which is certainly not as nice. (If you were expecting to see something like http://twistedmatrix.com/documents/10.1.0/core/howto/index.html, that's here: http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/core/howto/index.html .) As I said, the order can be changed manually. - The bottom of every page has a Download in other formats: Plain Text link that doesn't work; presumably scooped up from the Trac templates. It should be removed (the Sphinx Show Source link is in the right-hand sidebar anyway). Yeah, this is a flaw in the trac theme. Looks like I forgot to remove it. This seems like a simple fix - if you could do it quickly so we can see if fixes get propagated to the buildbot, that would be great (so we can get the build logistics out of the way for future, more complex issues). Hmmm...it won't be propagated to the buildbot automatically, since it's in the theme files in the lore2sphinx repo. From what exarkun said it sounds like it would require another manual checkout of that repo. I'll change it pretty soon, though. - When the breadcrumb navigation at the top of the page gets too long[1], it wraps to a second line, which looks kind of odd because the nice gradient background image loops. Changing the CSS background colour to be the same as the bottom of the gradient image would probably help. The breadcrumb names are also kinda weird, and very wordy. http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/projects/conch/howto/conch_client.html says Twisted Twisted Documentation Table of Contents Twisted Conch Documentation Twisted Documentation. This is altogether too many Twisteds ;-). It should really be more like 'Twisted Documentation Conch Writing a client with Twisted.Conch' Again: this is a new feature, so I'm not too concerned if we defer fixing this; go ahead if it's easy, but if it's hard, it's still better than the status quo, which is to say, not having any navigation at all. The breadcrumbs use document titles. Of course, in my opinion the word Twisted is used far too often in the document titles, which I think will be less necessary as we move the docs toward a cohesive whole, rather than a bunch of separate articles (which I see as a long-term, out-of-scope-for-this-discussion goal). - Chromium's error console reports: - A 404 error for /trac/chrome/common/js/jquery.js - A 404 error for /trac/chrome/common/js/trac.js - A 404 error for /trac/chrome/common/js/search.js - A 404 error for /builds/sphinx-html-15615/_static/dots.gif Ah! This explains why the search quit working. Not sure why these files are missing, Probably something to do with the custom theme. They're missing because this code is hosted on *buildbot*. twistedmatrix.com, not twistedmatrix.com. It's probably most expedient to just make those links include the netloc for now. facepalm of course. - The Twisted.Conch tutorial[1] has a number of things surrounded with double-backticks like ``Deferred`` or ``ClientTransport``. I'm not sure if that's deliberate or if it's markup gone horribly wrong. This is probably due to missing spaces before/after the markup in the rst source. One of those things that will need to be fixed manually. I spent a lot of time trying to fix this in lore2sphinx and when you fix it in one place, it breaks somewhere else. Right now the automated conversion is about as good as it is likely to get
Re: [Twisted-Python] PyCon extreme Twisted talk!
Another Twisted feature that I find amazingly useful is having multiple servers (e.g. email and web and telnet and ssh, etc.) in one process. Manhole (IMO) makes sense for any Twisted process that provides multiple services like this. Kevin Horn On Thu, Oct 21, 2010 at 7:56 PM, Laurens Van Houtven l...@laurensvh.bewrote: I would really like to see something where manhole makes sense as one of the iterative improvements, because it's a feature many people find very impressive. lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted.words.protocols.jabber and XEP-0138 stream compression
On Wed, Dec 15, 2010 at 11:13 AM, BURGUIERE Thomas thomas.burgui...@c-s.frwrote: Hey all, I've search (in Twisted 10.2.0) for a reference to XEP-0138 (stream compression) in twisted.words.protocols.jabber but I don't see anything referring to it. Is there a way to activate stream compression ? (I have a server which have a parameter for zlib stream compression) And if so what do i need to do ? Thanks, ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python No idea, but you might ask on the Twisted-Jabber list as well: https://mailman.ik.nu/mailman/listinfo/twisted-jabber Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Refactoring Documentation
, etc. in a way that doesn't require thousands of words of circular explanations, digressions, and duplications. - Document the different Twisted projects as what they are: really useful libraries and/or clients/servers built on top of Twisted that just so happens to also ship with the core. One of them is likely to do what the prospective user wants, too. There are also some things I *don't* want to accomplish, at least initially: - Explain asynchronous programming in depth - Get into the level of detail that the Krondo series does (but I plan to borrow from it!) There are things I'll *need help with*: - The final word on what is/isn't to be used in what context; practical, real-world explanations of why I would use A instead of B (not just it's more flexible). - Better documentation from project creators/maintainers/lovers on their project. These projects make a lot of hard stuff trivial and many do not convey that very well. I'll do my best to reorganize and improve the existing documentation, but most of the projects cater to a specific niche that requires its own core knowledge. Most of this was discussed with Jean-Paul and Glyph at tonight's Python meet-up in Cambridge. Some work has already begun on the new docs here: https://github.com/tdavis/twisted-docs I hope the above came off as optimistic and constructive as I meant it: it's not my intention to insult the documentation or those who wrote it. I just think with a dose of hindsight and a lot of effort, the docs can be awesome. And they would improve the entire Twisted ecosystem along with them. Cheers, Tom Tom, I very much like this idea, and in fact I have some ideas for this, but I've been putting off doing anything about it until the (neraly done! really!) conversion of the current documentation to Sphinx is complete (see ticket #4500). I got off track with that the last part of last year, but one of my New Year's Resolutions is to complete that project. How can we combine these efforts, or at least keep from working at cross purposes? I see from your link above that you are building your own Sphinx project. Perhaps you would be better off working from the results of the Lore2sphinx conversion? Are you modifying existing docs or working from scratch? Let's get together on this! Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Refactoring Documentation
reading. Would you volunteer to rewrite 28 thousand lines of code all at once? Or submit a 28 thousand line patch for review? As problematic as the current situation is, there is a definite potential for some baby vs. bathwater confusion in improving it. I am open to the idea that documentation needs different review practices from code, and there are things about documentation (like narrative flow) which are impossible to automatically verify and may require significant changes at multiple layers simultaneously. These features may not have an obvious parallel in code - which can typically be evolved incrementally, despite the occasional frustrated lament that everything needs to be rewritten. But, parts of the documentation are code. Another major problem with the current documentation is that it is out of date. We know when code goes out of date because it starts raising exceptions and emitting warnings from our test suite, but we only find out about documentation examples are going out of date when we do a release and some poor soul encounters an error when trying to follow along. I'd really like any effort that undertakes to substantially change the structure of the documentation to make a concerted effort to hit the ground running with test-driven examples which can be automatically verified, so that we have some idea of the impact of code changes on the docs. There's a ticket for writing tests for the code samples in Trac. Also, while you're energized now, please consider what happens to our documentation efforts if you run out of time or out of motivation halfway through this process. Incremental improvements may not necessarily provide the best outcome, but they _do_ mean that users get to see some results from these volunteer efforts even if the original volunteer gets busy with other aspects of life and can't complete their project. Many of your bullet-points above are well suited to individual efforts (some of which are actually issues with the web site, rather than the documentation per se). @Tom: agree here. Let's get _some_ benefit out of your energy, rather than you getting discouraged and having half a great set of docs floating out there someplace confusing users. I've been through this myself a couple of times (I've been pretty much ready to pull the trigger at least three times!) and have gotten discouraged and/or off track. After a while, my voom comes back, but in the meantime, the trunk has progressed, and there's that much more work to do. My advice is to try and get many small changes made, and get them _done_ rather than a few huge changes. I'd like to suggest that, at the very least, you create a branch of Twisted (perhaps on Launchpad?) and integrate your work into the doc/ hierarchy of the current tree. That way, at least, it would be easier for someone else to pick up where you've left off and fold things into the right place. This would also provide a way to more easily integrate the doc test suite into the main test suite on our existing build and release infrastructure. I hope the above came off as optimistic and constructive as I meant it: it's not my intention to insult the documentation or those who wrote it. I just think with a dose of hindsight and a lot of effort, the docs can be awesome. And they would improve the entire Twisted ecosystem along with them. Indeed, as the author of many of these original documents, I did not feel insulted in either my person or my work :). Hopefully you didn't feel that way either after reading this reply! I think this has been pretty constructive on both sides. I look forward to more. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Refactoring Documentation
I alluded in my earlier mail to some of the improvement ideas I have had regarding Twisted's docs (other than the conversion to Sphinx), and I though I'd elaborate Here's some things I had planned (yes, I'm using the term loosely) to try and add/improve/fix in the Twisted docs after the Sphinx conversion was finished: (urg, I know I've been keeping a list of these someplace, and some are in trac tickets, etc. but I can't find it right now, so I may ramble a bit) - Add a basic intro to Twisted. This would introduce some basic ideas similar to the krondoblog tutorial, but in less detail. - What is Twisted good for - Explain the most important parts of Twisted for people to get started with. IMO, this is a) the reactor, b) deferreds, c) some of the basic interfaces/classes, esp. Factories/Protocols (I'd like to see some docs on Transports as well) - more/better UDP stuff - How to test your Twisted apps (e.g., the idea of faking up a transport never occurred to me until I read a test that did it, and it's been one of the most useful techniques I've found) - Task-based docs - how to serve a web page, how to send a mail, how to write an IRC client (just to cut down on the questions!) etc. The basic stuff at first (what newbie's will be looking for), maybe eventually turning into a cookbook of sorts. - A revamped section on How to write twisted documentation, since that would likely be very different after the Sphinx conversion ( What Sphinx markup to use, and I also have some custom extensions, etc. which need to be documented). - How to _build_ the documentation - Beginner's tutorial - better glossary - move a bunch of stuff out of the Trac wiki, and into the Sphinx project. There's stuff there that has become de-facto documentation, which really needs to be version-controlled, etc. - install docs So after looking over Tom's skeleton Sphinx project and a night's sleep, I think we're pretty close to on the same page (or at least pages in the same book). It looks like a lot of this would be covered in Tom's Task-based docs and the Base Documentation section (Suiting Up, Diving in). So here's what I'm kind of thinking as far as combining efforts: 1) I'll continue with the Project Documentation conversion, while Tom works on the other bits. Should be fairly easy to combine them, I'm thinking. 2) Let's leave the Project Documentation pretty much as-is for the moment, until the Sphinx convo is done. 3) I wonder if at least some of the task-based docs shouldn't be put into the project sections, and then just linked to from the task-based section? Thoughts, Tom? [As an aside, is there any way to get the Combinator stuff from the old Trac wiki back online, or at least readable someplace? (Hrmmm, it looks like Google's cache has it for the moment). Also, it would be much easier to get new contributors, especially for Windows, if Combinator worked out of the box on Windows. There's (or were) 3 or 4 patches in the Divmod SVN repo that needed to be applied in order to get it to work, and now that the Divmod site is offline it's really hard to set it up on a new machine, even for me, and I have a working example to go from. Yeah it's off topic, sue me.] ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Flags with MaildirMailbox
On Sun, Jan 16, 2011 at 10:27 AM, goffi go...@goffi.org wrote: On 16/01/2011 01:44, exar...@twistedmatrix.com wrote: Not planned (planned in Twisted usually means there is a ticket for it). You're welcome to contribute that functionality, though! Do you have an idea about how it might fit into the existing code and API? Jean-Paul Well not really, but it's a good way to learn :). I'm really (really really) busy at the moment, so I probably can't in the next days, but I will try to have a look (and maybe turn on IRC/the documentation if I need help and review on the coding convention of Twisted). thanks for the reply :) When you get some time to look at it, I'd encourage you to at least file a ticket, so this issue doesn't get lost. Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Refactoring Documentation
On Thu, Jan 20, 2011 at 4:45 PM, exar...@twistedmatrix.com wrote:
On 10:05 pm, kevin.h...@gmail.com wrote:
On Thu, Jan 20, 2011 at 1:55 AM, Glyph Lefkowitz
gl...@twistedmatrix.comwrote:
(I was going to point you at the output of the buildbot, but it looks
like
something in trunk has broken the conversion process. Anyone know
what this
is about?
http://buildbot.twistedmatrix.com/builds/sphinx-
html-15724/contents.html)
I was going to address this as well. I've been seeing what are
apparently
intermittent problems with the buildbot, though I haven't seen that one
before, since the beginning of the year when I started seriously
looking at
this stuff again.
I was able to build the docs properly on the buildbot yesterday, and in
fact
couldn't replicate any of the issues I had previously seen (which, in
my
case, seem to mostly have to do with missing CSS files). when I was
talking
to exarkun about it on IRC yesterday.
I'm not sure whether it has to do with the buildbot configuration,
maybe
stuff timing out, network issues, problems in the actual conversion
process,
or what.
Does this help?
http://buildbot.twistedmatrix.com/builders/sphinx-
documentation/builds/123/steps/shell_4/logs/stdio
The salient point perhaps being:
File
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib.py,
line 347, in open_http
'got a bad status line', None)
IOError: ('http protocol error', 0, 'got a bad status line', None)
Jean-Paul
Hmmm...yes, I think it does. It looks like the l2s_builder script is
choking on fetching the DTD for some reason. lore2sphinx itself is supposed
to cache the DTD between runs, but I'm betting it doesn't work in the
buildbot due to the (hopefully) clean environment it has every time it runs.
I can change it to not care about that, but if I do, then it won't be able
to resolve HTML entities (of which there are a few in the Lore sources,
mostly em-dashes IIRC). So the XML parsing will choke on those, unless I
tell it to ignore them, in which case it might ignore other errors...etc.
I'll see what I can do to fix it, I haven't worked on that bit for a long
time. I'll see if possible to add a pre-cached DTD file to teh
lore2sphinx repo, and that should fix that problem at least.
Thanks for the research!
(I sometimes forget that buildbot keeps a log of the stdio. You wouldn't
think that I would, since I use the analogous logs in hudson fairly
frequently, but I do. Sigh.)
Kevin Horn
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Refactoring Documentation
On Thu, Jan 20, 2011 at 5:57 PM, Tim Allen screwt...@froup.com wrote: On Thu, Jan 20, 2011 at 05:22:55PM -0600, Kevin Horn wrote: Hmmm...yes, I think it does. It looks like the l2s_builder script is choking on fetching the DTD for some reason. lore2sphinx itself is supposed to cache the DTD between runs, but I'm betting it doesn't work in the buildbot due to the (hopefully) clean environment it has every time it runs. I can change it to not care about that, but if I do, then it won't be able to resolve HTML entities (of which there are a few in the Lore sources, mostly em-dashes IIRC). You mean these DTDs? twisted/lore/xhtml1-strict.dtd twisted/lore/xhtml1-transitional.dtd They reference the xhtml-*.ent entity definitions which are also in the same directory. It would be neat if lore2sphinx could be taught to use the DTDs packaged with lore instead of having to download them from the Internet every time. Huh. Never even knew that was there. It probably could, and the reason it downloads from the internet was because that's the default way of doing it in lxml. I've since figured out how to override that behavior (which is how the caching works) so maybe that wouldn't even be hard. The easiest/fastest fix for the moment though would probably be to pre-populate the cache as I mentioned before, since IIRC, this would just involve adding the file to my hg repo. I'll have to look into it though, it may be just as easy to do it the other way, though I don't want to depend necessarily on having Twisted's code available (remember, this is supposed to work on the various divmod projects, and anything else that uses Lore, too). Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Refactoring Documentation
On Thu, Jan 20, 2011 at 6:06 PM, exar...@twistedmatrix.com wrote: On 20 Jan, 11:57 pm, screwt...@froup.com wrote: On Thu, Jan 20, 2011 at 05:22:55PM -0600, Kevin Horn wrote: Hmmm...yes, I think it does. It looks like the l2s_builder script is choking on fetching the DTD for some reason. lore2sphinx itself is supposed to cache the DTD between runs, but I'm betting it doesn't work in the buildbot due to the (hopefully) clean environment it has every time it runs. I can change it to not care about that, but if I do, then it won't be able to resolve HTML entities (of which there are a few in the Lore sources, mostly em-dashes IIRC). You mean these DTDs? twisted/lore/xhtml1-strict.dtd twisted/lore/xhtml1-transitional.dtd They reference the xhtml-*.ent entity definitions which are also in the same directory. It would be neat if lore2sphinx could be taught to use the DTDs packaged with lore instead of having to download them from the Internet every time. It could even use Lore's parser, twisted.lore.tree.parseFileAndReport ;) Jean-Paul Yes, but: - The docs (well, docstrings) aren't very clear about exactly what that does, or what quirks it might or might not have. - Is the DOM document it returns a typical XHTML DOM document? Is there anything special/different about it since it's a Lore XHTML doc? - Consider the number of tickets open involving Lore's parser (or at least there were several when I started this whole business). - The whole point of this project is to stop using (and by extension supporting/maintaining) Lore, so it seems kind of backwards to me to depend on it. - I'd rather deal with an etree-a-like, than a straight DOM document (admittedly, personal preference) - I certainly don't want to rewrite everything at this point ;) Kevin Horn ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
