[Twisted-Python] deprecate using paths with trial

[Jean-Paul Calderone]

Hello,

Since trial has stopped adding the working directory to sys.path, it seems
very sensible to also stop accepting paths as a way to specify what tests
to run.  Instead, only package, module, class, or test method names should
be accepted.

Please add your thoughts (reasoned arguments, not +1s and -1s) to this
ticket:

https://twistedmatrix.com/trac/ticket/9035

Thanks,
Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 17.1 Release Announcement

[Jean-Paul Calderone]

On Sat, Feb 11, 2017 at 5:03 AM, Amber Hawkie Brown <
hawk...@atleastfornow.net> wrote:

> On behalf of Twisted Matrix Laboratories, I am honoured to announce the
> release of Twisted 17.1!
>
> The highlights of this release are:
>
> - twisted.web.client.Agent now supports IPv6! It's also now the primary
> web client in Twisted, with twisted.web.client.getPage being deprecated in
> favour of it and Treq.
> - twisted.web.server has had many cleanups revolving around timing out
> inactive clients.
> - twisted.internet.ssl.CertificateOptions has had its `method` argument
> deprecated, in favour of the new raiseMinimumTo, lowerMaximumSecurityTo,
> and insecurelyLowerMinimumTo arguments, which take TLSVersion arguments.
> This allows you to better give a range of versions of TLS you wish to
> negotiate, rather than forcing yourself to any one version.
> - twisted.internet.ssl.CertificateOptions will use OpenSSL's
> MODE_RELEASE_BUFFERS, which will let it free unused memory that was held by
> idle TLS connections.
> - You can now call the new twist runner with `python -m twisted`.
> - twisted.conch.ssh now has some ECDH key exchange support and supports
> `hmac-sha2-384`.
> - Better Unicode support in twisted.internet.reactor.spawnProcess,
> especially on Windows on Python 3.6.
> - More Python 3 porting in Conch, and more under-the-hood changes to
> facilitate a Twisted-wide jump to new-style classes only on Python 2 in
> 2018/2019. This release has also been tested on Python 3.6 on Linux.
> - Lots of deprecated code removals, to make a sleeker, less confusing
> Twisted.
> - 60+ closed tickets.
>
> For more information, check the NEWS file (link provided below).
>
> You can find the downloads at  (or
> alternatively ). The NEWS
> file is also available at  twisted/blob/twisted-17.1.0/NEWS>.
>
> Many thanks to everyone who had a part in this release - the supporters of
> the Twisted Software Foundation, the developers who contributed code as
> well as documentation, and all the people building great things with
> Twisted!
>
>
Thanks all!  Great to see the troubles of 16.7 mostly behind us.

I do hope there will be at 17.1.1 soon for <
https://twistedmatrix.com/trac/ticket/9031> and particularly <
https://twistedmatrix.com/trac/ticket/9032>.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] SMTP: Authenticating on Outbound emails only

[Jean-Paul Calderone]

On Thu, Feb 9, 2017 at 12:29 PM, Anthony Lukach 
wrote:

> I am working on putting together an SMTP server implemented within
> Twisted.  This will act as a conduit to my API, where there are two basic
> flows:
>
> 1) A device that supports sending email can "send" an email through the
> SMTP server. This is, in effect, the SMTP server handling the message as an
> outbound request.  The device would authenticate with the server and then
> provide it the message to be sent (which in reality will be uploaded to my
> API).
>
> 2) A device can send an email to my SMTP server via their own SMTP server.
> My server would receive the incoming message, parse its contents, and then
> upload the data to my API. Naturally, these incoming messages would not be
> required to authenticate with the server.
>
> I'm having trouble constructing the server in a way that outbound messages
> require authentication but incoming messages do not.
>

twisted.mail and cred support anonymous access.  I think that what you want
is to implement an avatar that can only accept messages for local delivery
and use that for anonymous users.  If an anonymous user tries to send mail
to a non-local user, they get back an error.  Either they made a mistake
and they should try again after authenticating or they're trying to abuse
the service and that's what you want.  That's case (2).  Then implement the
relay logic in another avatar and use that for authenticated users.  That's
case (1).

This relies on the fact that the realm you supply is responsible for
creating avatars and the realm gets told the avatarId for which it should
create an avatar - and the anonymous user can be differentiated from other
users by the avatarId.

Does that help?


> Can anyone direct me to an example of an SMTP server that is both
> accepting inbound and outbound emails and authenticates only on outbound
> emails?
>
>
As it happens, yes...

This avatar supports local-only delivery:
https://github.com/twisted/quotient/blob/master/xquotient/mail.py#L51
This one supports relaying:
https://github.com/twisted/quotient/blob/master/xquotient/mail.py#L528
Here's the factory and portal setup code:
https://github.com/twisted/quotient/blob/master/xquotient/mail.py#L302

The realm is a bit spread out and involves some Axiom-specific concepts
that you probably don't need to know... But let me know if the above three
links don't make things clear.

 Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] "twistd" in Twisted 16.4.0 can't import modules/packages from current working directory

[Jean-Paul Calderone]

On Fri, Jan 13, 2017 at 1:13 AM, Glyph Lefkowitz 
wrote:

> Thanks for highlighting those.  I've put the link in the other direction
> as well.
>
>
Craig seems eager to go ahead with reverting this change in behavior.

https://github.com/twisted/twisted/pull/672#issuecomment-275956265

As far as I can tell, no one has weighed in on the other side.  So I'm
inclined to go along with the reversion.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] web client FileBodyProducer - transfer encoding

[Jean-Paul Calderone]

On Mon, Jan 30, 2017 at 7:44 PM, Glyph Lefkowitz 
wrote:

> Gotcha.  I guess what I meant was that you shouldn't care about this at
> the application level, but you're talking about an operational concern, not
> an application-level concern.
>
> Perhaps this should be a tunable on Agent somehow.  Can you file a ticket?
>


Well... It *is* sort of tuneable, as you implied earlier.

If you pass an IBodyProducer with a non-None length, Agent will send a
Content-Length header - not use chunked Transfer-Encoding.
FileBodyProducer doesn't know how to determine the length of a StringIO, so
you get chunked with this example.  If you write the JSON to a regular file
and FileBodyProducer(open(the file)) you'll get a Content-Length request.
You could also write a new (trivial) IBodyProducer that does know how to
compute the length of a StringIO.

The documentation doesn't exactly spell this out - but the only reason
`length` is part of the interface is to be able to generate the
Content-Length header.

What would the ticket be?  Expanded documentation to make this behavior
into an explicit guarantee of the interface?  A new toggle somewhere to
force Agent.request into one mode or the other (regardless of the
performance consequences)?

And that stuff you said about proxies before was true ... So even if you
can control this in Agent, you're still not *guaranteed* the server will
see what you send.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Can't create ticket in Trac

[Jean-Paul Calderone]

On Fri, Jan 27, 2017 at 7:25 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Jan 27, 2017, at 4:16 PM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
>
> On Fri, Jan 27, 2017 at 7:09 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>>
>> On Jan 27, 2017, at 3:53 PM, Jean-Paul Calderone <
>> exar...@twistedmatrix.com> wrote:
>>
>>
>> My browser is not using HTTP/2 to talk to twistedmatrix.com (idnar
>> suggests this is because lack of ALPN in twistedmatrix.com's OpenSSL and
>> lack of NPN in Chrome).
>>
>>
>> It might be interesting to upgrade to Twisted trunk to see if some of the
>> HTTP server bugfixes that have landed will address this; however, I can't
>> come up with a coherent theory as to why certain POSTs would just hang.
>>
>>
> I've yet to detect any pattern to the problem.  With no way to reliably
> reproduce the problem and not other real metric for whether things are
> getting better or worse, I don't think I could evaluate the results of any
> changes.
>
>
> How often do you experience it?  Is it often enough that we could, say,
> check back in in a week?
>

My dataset is pretty small...  Perhaps 6 or 7 times in the last 2 or 3
weeks?  I don't know how many POSTs I've done in that time.  Maybe a couple
dozen?  I could probably tell the difference if it were *completely* fixed
(or *completely* broken :) so it may be worth making the change on that
basis.

Jean-Paul



>
> -glyph
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Can't create ticket in Trac

[Jean-Paul Calderone]

On Fri, Jan 27, 2017 at 7:09 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Jan 27, 2017, at 3:53 PM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
>
>
> My browser is not using HTTP/2 to talk to twistedmatrix.com (idnar
> suggests this is because lack of ALPN in twistedmatrix.com's OpenSSL and
> lack of NPN in Chrome).
>
>
> It might be interesting to upgrade to Twisted trunk to see if some of the
> HTTP server bugfixes that have landed will address this; however, I can't
> come up with a coherent theory as to why certain POSTs would just hang.
>
>
I've yet to detect any pattern to the problem.  With no way to reliably
reproduce the problem and not other real metric for whether things are
getting better or worse, I don't think I could evaluate the results of any
changes.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Can't create ticket in Trac

[Jean-Paul Calderone]

On Fri, Jan 27, 2017 at 6:52 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Jan 27, 2017, at 3:49 PM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
>
> On Fri, Jan 27, 2017 at 6:21 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
>  wrote:
>
>>
>> On Jan 27, 2017, at 3:12 PM, Craig Rodrigues <rodr...@crodrigues.org>
>> wrote:
>>
>>
>>
>> On Fri, Jan 27, 2017 at 2:29 PM, Glyph Lefkowitz <gl...@twistedmatrix.com
>> > wrote:
>>
>>>
>>> On Jan 27, 2017, at 2:03 PM, Craig Rodrigues <rodr...@crodrigues.org>
>>> wrote:
>>>
>>> Hi,
>>>
>>> I can't seem to create a ticket in Trac.  Can someone look?
>>>
>>> I can create tickets.  What behavior do you see?
>>>
>>>
>> After entering all the fields to create the ticket, and then clicking
>> submit, it would just hang
>> without creating the ticket.
>>
>>
>> Forever? No timeout?  Just the HTTP POST never returns?
>>
>>
> FWIW, I've been experiencing similar problems.  In my case, the POST
> doesn't return for as long as I care to wait (in one case, around 20
> minutes).  Re-submitting, sometimes 3-5 times, generally results in a
> successful eventually.  This is with Chrome Version 54.0.2840.90 (64-bit)
> on Ubuntu 16.10.
>
>
> I'm using Safari 10.0.3 (12602.4.8) on Sierra.
>
> Interesting to check: I'm using HTTP/2 to talk to twistedmatrix.com.  Are
> either of you?  I wonder if this is a bug in the HTTP/2 stack or perhaps
> one of the HTTP/1.1 changes made to accommodate the HTTP/2 implementation.
>

My browser is not using HTTP/2 to talk to twistedmatrix.com (idnar suggests
this is because lack of ALPN in twistedmatrix.com's OpenSSL and lack of NPN
in Chrome).

Jean-Paul

>
> -glyph
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Can't create ticket in Trac

[Jean-Paul Calderone]

On Fri, Jan 27, 2017 at 6:21 PM, Glyph Lefkowitz 
wrote:

>
> On Jan 27, 2017, at 3:12 PM, Craig Rodrigues 
> wrote:
>
>
>
> On Fri, Jan 27, 2017 at 2:29 PM, Glyph Lefkowitz 
> wrote:
>
>>
>> On Jan 27, 2017, at 2:03 PM, Craig Rodrigues 
>> wrote:
>>
>> Hi,
>>
>> I can't seem to create a ticket in Trac.  Can someone look?
>>
>> I can create tickets.  What behavior do you see?
>>
>>
> After entering all the fields to create the ticket, and then clicking
> submit, it would just hang
> without creating the ticket.
>
>
> Forever? No timeout?  Just the HTTP POST never returns?
>
>
FWIW, I've been experiencing similar problems.  In my case, the POST
doesn't return for as long as I care to wait (in one case, around 20
minutes).  Re-submitting, sometimes 3-5 times, generally results in a
successful eventually.  This is with Chrome Version 54.0.2840.90 (64-bit)
on Ubuntu 16.10.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Problems with inlineCallback, Deferred, yield and Python 3 in buildbot

[Jean-Paul Calderone]

On Mon, Jan 23, 2017 at 7:08 PM, Craig Rodrigues 
wrote:

>
>
> On Sat, Jan 21, 2017 at 8:06 PM, Glyph Lefkowitz 
> wrote:
>
>>
>> On Jan 21, 2017, at 6:15 PM, Craig Rodrigues 
>> wrote:
>>
>> If I run the test on Python 2, I don't get the error, and on line 93,
>> brdicts is a dict.
>> However, if I run the test on Python 3, brdicts is a Deferred, thus the
>> error.
>>
>>
>> This really ought to be impossible.  If I were seeing this,
>> single-stepping through inlineCallbacks in pudb would probably be my next
>> step.  Have you made any attempts to simplify it down to remove some of the
>> buildbot-specific code?
>>
>> -glyph
>>
>>
>
> I did some more debugging.  The callstack is quite deep. :/
> I used this command to trigger the error:
>
> trial  buildbot.test.unit.test_process_buildrequestdistributor.
> TestMaybeStartBuilds.test_slow_db
>
> I found in this function in buildbot's BuildRequestsEndpoint.get()
> function ( https://github.com/buildbot/buildbot/blob/master/master/
> buildbot/data/buildrequests.py#L146 ) there is this line:
>
>
>
> *defer.returnValue([(yield self.db2data(br)) for br in
> buildrequests])*
> On Python 2, this line returns:
>   an object list
>   each list entry is a dictionary of name/value pairs that looks like:
>
> [{'buildrequestid': 10, 'complete': False, 'waited_for': False,
> 'claimed_at': None, 'results': -1, 'claimed': False, 'buildsetid': 11,
> 'complete_at': None, 'submitted_at': datetime.datetime(1970, 1, 2, 12, 6,
> 40, tzinfo=tzutc()), 'builderid': 77, 'claimed_by_masterid': None,
> 'priority': 0}, {'buildrequestid': 11, 'complete': False, 'waited_for':
> False, 'claimed_at': None, 'results': -1, 'claimed': False, 'buildsetid':
> 11, 'complete_at': None, 'submitted_at': datetime.datetime(1970, 1, 2, 13,
> 30, tzinfo=tzutc()), 'builderid': 77, 'claimed_by_masterid': None,
> 'priority': 0}]
>
> On Python 3, this returns:
>   an object  locals>.
>   of type 
>
>
Yep.

On Python 2, [(yield self.db2data(br)) for br in buildrequests] is a list
comprehension.  It will have len(buildrequests) elements and each will be
the value sent back in to the generator via the yield expression.

On Python 3, the same expression is a list of one element which is a
generator expression.

This form is much clearer, I think, and behaves as intended on both
versions of Python:

results = []
for br in buildrequests:
results.append((yield self.db2data(br)))
defer.returnValue(results)

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] admin/pr_as_branch

[Jean-Paul Calderone]

On Sun, Jan 22, 2017 at 8:31 PM, Jean-Paul Calderone <
exar...@twistedmatrix.com> wrote:

> On Sun, Jan 22, 2017 at 8:28 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>>
>> On Jan 22, 2017, at 5:24 PM, Jean-Paul Calderone <
>> exar...@twistedmatrix.com> wrote:
>>
>> On Sun, Jan 22, 2017 at 8:19 PM, Glyph Lefkowitz <gl...@twistedmatrix.com
>> > wrote:
>>
>>>
>>> On Jan 22, 2017, at 5:15 PM, Jean-Paul Calderone <
>>> exar...@twistedmatrix.com> wrote:
>>>
>>> Hello,
>>>
>>> I didn't find any hints about the workflow surrounding the
>>> admin/pr_as_branch tool so I invented one and wrote it up on the wiki:
>>>
>>> https://twistedmatrix.com/trac/wiki/ReviewProcess?action=diff=76
>>>
>>> Jean-Paul
>>>
>>>
>>> Thanks for writing this up.  The workflow is evolving organically.
>>>
>>> However, the "close the PR" step confuses me.  If you do this, the
>>> original contributor won't be able to respond to feedback.  What is the
>>> desired effect of this second PR?
>>>
>>
>>
>> The idea I heard is that it provides a place to hang build
>> failure-related review comments.  I also had some vague notion that it
>> would be the place you'd look to see the complete CI results.
>>
>> So.  Where should further reviews go and where do you find CI results, if
>> you don't create a new PR?
>>
>>
>> You find the CI results on the existing PR; they'll be displayed there.
>> You just need to make sure that a branch in the repo has the exact same
>> commit ID when you push it as the PR has.  (Statuses are reported on
>> commits, and triggered on pushes, so everything should just line up.)
>>
>>
> After the branch in the repo is all set (which I assume pr_as_branch will
> do), does CI notice without further help and start the necessary builds?
>
>
>> As far as further reviews - on the existing PR too, same as usual.
>>
>> You will need to run the script again when the contributor pushes their
>> response to a review.  One feature that would be useful is to not need to
>> specify the branch name twice, but if you name the second pr_as_branch
>> invocation differently, it just means one more branch that should be
>> deleted when looking at git branch --merged.
>>
>>
> Yep, that makes sense.
>

Updated:
https://twistedmatrix.com/trac/wiki/ReviewProcess?action=diff=77_version=75



>
> Jean-Paul
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] admin/pr_as_branch

[Jean-Paul Calderone]

On Sun, Jan 22, 2017 at 8:28 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Jan 22, 2017, at 5:24 PM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
>
> On Sun, Jan 22, 2017 at 8:19 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>>
>> On Jan 22, 2017, at 5:15 PM, Jean-Paul Calderone <
>> exar...@twistedmatrix.com> wrote:
>>
>> Hello,
>>
>> I didn't find any hints about the workflow surrounding the
>> admin/pr_as_branch tool so I invented one and wrote it up on the wiki:
>>
>> https://twistedmatrix.com/trac/wiki/ReviewProcess?action=diff=76
>>
>> Jean-Paul
>>
>>
>> Thanks for writing this up.  The workflow is evolving organically.
>>
>> However, the "close the PR" step confuses me.  If you do this, the
>> original contributor won't be able to respond to feedback.  What is the
>> desired effect of this second PR?
>>
>
>
> The idea I heard is that it provides a place to hang build failure-related
> review comments.  I also had some vague notion that it would be the place
> you'd look to see the complete CI results.
>
> So.  Where should further reviews go and where do you find CI results, if
> you don't create a new PR?
>
>
> You find the CI results on the existing PR; they'll be displayed there.
> You just need to make sure that a branch in the repo has the exact same
> commit ID when you push it as the PR has.  (Statuses are reported on
> commits, and triggered on pushes, so everything should just line up.)
>
>
After the branch in the repo is all set (which I assume pr_as_branch will
do), does CI notice without further help and start the necessary builds?


> As far as further reviews - on the existing PR too, same as usual.
>
> You will need to run the script again when the contributor pushes their
> response to a review.  One feature that would be useful is to not need to
> specify the branch name twice, but if you name the second pr_as_branch
> invocation differently, it just means one more branch that should be
> deleted when looking at git branch --merged.
>
>
Yep, that makes sense.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] admin/pr_as_branch

[Jean-Paul Calderone]

On Sun, Jan 22, 2017 at 8:19 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Jan 22, 2017, at 5:15 PM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
>
> Hello,
>
> I didn't find any hints about the workflow surrounding the
> admin/pr_as_branch tool so I invented one and wrote it up on the wiki:
>
> https://twistedmatrix.com/trac/wiki/ReviewProcess?action=diff=76
>
> Jean-Paul
>
>
> Thanks for writing this up.  The workflow is evolving organically.
>
> However, the "close the PR" step confuses me.  If you do this, the
> original contributor won't be able to respond to feedback.  What is the
> desired effect of this second PR?
>


The idea I heard is that it provides a place to hang build failure-related
review comments.  I also had some vague notion that it would be the place
you'd look to see the complete CI results.

So.  Where should further reviews go and where do you find CI results, if
you don't create a new PR?

Jean-Paul



>
> -glyph
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] admin/pr_as_branch

[Jean-Paul Calderone]

Hello,

I didn't find any hints about the workflow surrounding the
admin/pr_as_branch tool so I invented one and wrote it up on the wiki:

https://twistedmatrix.com/trac/wiki/ReviewProcess?action=diff=76

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Strange ImportError after switching from OS package to pip

[Jean-Paul Calderone]

On Tue, Jan 17, 2017 at 12:53 PM, Роман Мещеряков <
romanmescherya...@yandex.ru> wrote:

> *Hi everyone! The 14.0.2 version of Twisted is latest that is available
> for Raspbian as a package, so until today I used it. I run my Twisted
> application in the following way:*
>
>
> *twistd -ny master_player2.tac*
>
>
> *There were no problems with starting my application.*
>
> *Yesterday I started facing strange problem with unjellying which looked
> like a bug. So I decided to remove old package version of Twisted
> (installed using apt-get) and install latest version using pip (which was
> 16.6.0). The commands I issued were (typing by memory):*
>
>
Seems likely related to
http://twistedmatrix.com/pipermail/twisted-python/2017-January/031021.html

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] "twistd" in Twisted 16.4.0 can't import modules/packages from current working directory

[Jean-Paul Calderone]

Note there is still some confusion over this matter.  See <
http://twistedmatrix.com/trac/ticket/8972>, <
http://twistedmatrix.com/trac/ticket/8978>, and <
https://github.com/twisted/twisted/pull/672>.

Jean-Paul

On Tue, Jan 3, 2017 at 3:24 PM, Jean-Paul Calderone <
exar...@twistedmatrix.com> wrote:

> On Tue, Jan 3, 2017 at 3:22 PM, Amber "Hawkie" Brown <
> hawk...@atleastfornow.net> wrote:
>
>>
>>
>> We have, in the past, fixed up historic NEWS files in later releases
>> (e.g. the one which removed 2.6 support). We could always roll these
>> changes into a post1, but, that seems like a lot of effort. Maybe we could
>> put errata on the blog?
>>
>>
> With a link to the blog post in the NEWS file?  Cold checking blog was
> below "read git history and try to correlate with the issue tracker" on my
> list of how to track down this change.
>
> Jean-Paul
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Testing Strategies (was Re: Streaming Requests)

[Jean-Paul Calderone]

On Wed, Jan 11, 2017 at 11:41 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Jan 9, 2017, at 4:13 AM, Jean-Paul Calderone <exar...@twistedmatrix.com>
> wrote:
>
> On Mon, Jan 9, 2017 at 4:52 AM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>> On a related note, 'proto_helpers' is in a super awkward place.
>>  'twisted.testing', anyone? :)
>>
>
> Yes.  I almost suggested this about a week ago when I was preparing to
> contribute some testing code but then realized I couldn't contribute the
> code after all. :(
>
>
> Well, now it's just a matter of time :).
>
>
Some further thoughts on this... What I've actually been doing elsewhere (
*mostly*) is to put the testing implementations quite near the real
implementations.  For example, I recently introduced
`workproject.subscription_manager.{network_client,memory_client}`.  But
contrary to this, in txAWS I'm following the pre-existing pattern which is
more like `txaws.service.AWSServiceRegion.get_ec2_client` &
`txaws.testing.service.FakeAWSServiceRegion.get_ec2_client`.

Which I prefer, I'm not really sure yet.  Role-named modules are easy to
recognize.  On the other hand, testing code next to implementation code is
easier to stumble across and you don't *have* to recognize a separate
testing module to find it.  Also, role-named modules tend to pile up and
get in each other's way (start working with `twisted.testing`,
`txaws.testing`, etc and things start to get confusing quickly).

Additionally, a related idea is that often the testing implementation might
actually serve as a good starting place for the real implementation.  I
think the code in txAWS is a good example of this (ie, the real and fake
implementations duplicate a lot of logic with some pointless divergences).
In the olden days I would have said "the testing implementation is often a
good *base class* for the real implementation".  But now I'll mumble
something about composition.  Put another way:

   - Clock should be the real implementation of all of the time-source
   independent logic of IReactorTime
   - FakeAWSServiceRegion should be the real implementation of the "client
   factory" logic (get_XXX_client) of AWSServiceRegion

and so on.

Perhaps this also speaks to the idea you mentioned that there should be a
stack of thin abstractions but I snipped that content already so I'll try
responding to that separately, later.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Announcing txAWS 0.2.3.1

[Jean-Paul Calderone]

I've just release txAWS 0.2.3.1.  txAWS is a library for interacting with
Amazon Web Services (AWS) using Twisted.

AWSServiceEndpoint's ssl_hostname_verification's parameter now defaults to
True instead of False.  This affects all txAWS APIs which issue requests to
AWS endpoints.  For any application which uses the default
AWSServiceEndpoints, the server's TLS certificate will now be verified.

This resolves a security issue in which txAWS applications were vulnerable
to man-in-the-middle attacks which could either steal sensitive information
or, possibly, alter the AWS operation requested.

The new release is available on PyPI in source and wheel forms.  You can
also find txAWS at its new home on github, .

Special thanks to Least Authority Enterprises
() for
sponsoring the work to find and fix this issue and to publish this new
release.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Testing Strategies (was Re: Streaming Requests)

[Jean-Paul Calderone]

On Mon, Jan 9, 2017 at 4:52 AM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

> On Jan 8, 2017, at 4:34 PM, Jean-Paul Calderone <exar...@twistedmatrix.com>
> wrote:
>
>
> Here's one example I know of off the top of my head, <
> https://github.com/ScatterHQ/flocker/blob/master/flocker/
> restapi/testtools.py#L316>.  This one isn't a from-scratch
> re-implementation of the implicit Request interface but a Request subclass,
> instead.  However, it still interacts with a lot of important pieces of
> Request which aren't part of IRequest.
>
>
> Mark already addressed how he won't be breaking this use-case (which is
> hugely important, and core to the whole idea of a compatibility policy, so
> that is as it should be).
>
> However, this kind of test-mocking is, I think, ultimately done at the
> wrong layer.  It's trying to override some very vaguely-specified internals
> in the middle of an implementation.
>
>
Absolutely.



> Really, twisted.web should provide its own testing tools, of course.  But
> if you're going to implement something that does this sort of overriding, I
> think the idiom to follow would be treq.testing: https://github.
> com/twisted/treq/blob/fcf5deb976c955ca6ef6484f414d25
> 839932940e/src/treq/testing.py, rather than any of the various
> implementations of DummyRequest (including more than a few I'm sure I've
> written).
>

Though, note, the link I gave above was support code for something very
similar to this treq code:
https://github.com/ScatterHQ/flocker/blob/master/flocker/restapi/testtools.py#L460

To clarify your point a bit (I think):


   - MemoryAgent (from Flocker) provides a testing IAgent by implementing
   an IRequest that does in-memory resource traversal to dispatch requests and
   generate responses.
   - RequestTraversalAgent (from treq) provides a testing IAgent by
   implementing (using) iosim to do in-memory protocol/transport interacts to
   drive an in-memory HTTP conversation that runs all of the regular Twisted
   Web HTTP processing machinery.

RequestTraversalAgent's approach is better because the protocol/transport
interface is better defined.  Because it's better
defined, RequestTraversalAgent doesn't have to touch pieces that we might
want to consider implementation details (whether they're _-prefixed or
not).  It also invokes a larger portion of the real implementation code
making it more likely to be a realistic simulation of real-world use of the
code.

Having spelled this out, what occurs to me now is
that RequestTraversalAgent is really just a step or so up the ladder and
there's further to go.  For example, RequestTraversalAgent only needs to
exist at all because `iosim` has a distinct interface.  This distinct
interface means you need a RequestTraversalAgent-like thing for each
reactor-using thing for which you want to provide a test double.  If this
adaption behavior were bundled up differently then I think we'd get a lot
more in-memory test doubles for free (or closer to it - we'd be another
rung up the ladder, at least).  That seems like it would be a big win given
how *few* of these testing helpers Twisted has historically managed to
provide (suggesting it's just too hard to do so given the current tools).


>
> This is a set of ideas that I've been gradually arriving at over a long
> period of time, but it's probably high time for some public discussion by
> now, even if it's just everybody saying "yeah, that sounds good" :-).
>

So, it pretty much sounds good, though with the above refinements. :)


>
> On a related note, 'proto_helpers' is in a super awkward place.
>  'twisted.testing', anyone? :)
>

Yes.  I almost suggested this about a week ago when I was preparing to
contribute some testing code but then realized I couldn't contribute the
code after all. :(

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Streaming Requests

[Jean-Paul Calderone]

On Sun, Jan 8, 2017 at 1:44 AM, Mark Williams 
wrote:

> * What?
>   A new year means renewed ambition.  So let's talk about receiving
>   streaming requests!
>
>
Hi Mark,

Thanks for tackling this long-standing issue!

I want to start off by responding to just one part of your email.  This is
actually the only part that looks potentially problematic to me so far,
which I take to be a good sign. :)


>   He also noted that the Request interface with which HTTPChannel
>   interacts is mostly not described by twisted.iweb.IRequest.  That
>   means we can augment the ways HTTPChannel talks to Request-like
>   things without breaking many public APIs.
>
>
How will this affect applications which have implemented Request-alike
classes (for example, for testing purposes)?  The interface may not be
explicitly defined as a Zope Interface right now but it is defined by the
current implementation and it is largely public (ie, non-_-prefixed
names).  Tests may have hooked in to parts of Request not documented by
IRequest out of necessity.  It would be a shame to break such carefully
crafted automated tests.

Here's one example I know of off the top of my head, <
https://github.com/ScatterHQ/flocker/blob/master/flocker/restapi/testtools.py#L316>.
This one isn't a from-scratch re-implementation of the implicit Request
interface but a Request subclass, instead.  However, it still interacts
with a lot of important pieces of Request which aren't part of IRequest.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] "twistd" in Twisted 16.4.0 can't import modules/packages from current working directory

[Jean-Paul Calderone]

On Tue, Jan 3, 2017 at 3:22 PM, Amber "Hawkie" Brown <
hawk...@atleastfornow.net> wrote:

>
>
> We have, in the past, fixed up historic NEWS files in later releases (e.g.
> the one which removed 2.6 support). We could always roll these changes into
> a post1, but, that seems like a lot of effort. Maybe we could put errata on
> the blog?
>
>
With a link to the blog post in the NEWS file?  Cold checking blog was
below "read git history and try to correlate with the issue tracker" on my
list of how to track down this change.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] "twistd" in Twisted 16.4.0 can't import modules/packages from current working directory

[Jean-Paul Calderone]

On Thu, Sep 1, 2016 at 11:36 AM, Amber "Hawkie" Brown <
hawk...@atleastfornow.net> wrote:

>
> > On 1 Sep 2016, at 21:42, Yuri  wrote:
> >
> > Hi all
> >
> > I couldn't find Twisted-specific group, so posting here.
> >
> > Recently Twisted 16.4.0 got released. Yesterday I've tried to upgrade it
> for my apps and got an error.
> >
> > ...
> >
> > The errors comes down to this: twistd script does not add current
> working directory to python path (or removes it, I don't know what exactly
> happens), so it fails to import any packages/modules from it. The issue
> does not appear in previous version (Twisted 16.3.2).
> >
> > Any ideas what caused it?
>
> Yes -- we moved to using setuptools console scripts, and these console
> scripts don't add "." to the PYTHONPATH. We realised this in prerelease but
> decided against fixing it, as it adding the current working dir to the PATH
> has lead to a lot of  subtle bugs in the past and this is a good chance to
> make a break from them.
>
> So, in short, this is expected behaviour -- we generally want people to be
> running twistd, trial, etc on *installed* Python packages -- testing or
> running from checkouts often hides many bugs about what is or isn't
> included in the installed package by accident. If you rely on this
> behaviour, though, set the PYTHONPATH environment variable to "." -- e.g.
> `env PYTHONPATH=. twistd -n myplugin`.
>
>
FWIW, as a user, it would have been nice to have a NEWS entry for this.  It
would have been easier to discover there than searching through the mail
archives.

Thanks,
Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] tx github announcements in #twisted-dev

[Jean-Paul Calderone]

How do folks feel about this?  Good idea or not?

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 16.7.0rc1 Release Candidate Announcement

[Jean-Paul Calderone]

On Fri, Dec 23, 2016 at 4:22 PM, Glyph Lefkowitz 
wrote:

>
>
> It's merged now: https://twistedmatrix.com/trac/ticket/8911#comment:5
>
> On a related note - this still linked the '#' syntax back to the ticket
> number in Trac.
>
> Perhaps related - tomprince fixed the bugs that were preventing us from
> upgrading us to 1.2, and upgraded us to 1.2.  So perhaps this has
> introduced a problem :).
>
> -glyph
>
>
It looks like the link to ticket 599 in the merge/resolve comment on ticket
8911 is a different kind of link than the one I was looking in to.  I agree
it would be nice to fix this, too, and perhaps I'll take a look.  My goal
was only to stop ticket 599 (for example) from also receiving a copy of
this comment - which appears to have succeeded.  Let me know if I've missed
something.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] AWS Auth v4 for Twisted?

[Jean-Paul Calderone]

Seeking a library that implements this for use with txAWS.  More details:

https://stackoverflow.com/questions/41322251/is-there-an-aws-auth-v4-implementation-usable-with-a-twisted-friendly-http-clien

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] trac deployment (was: memory leaks)

[Jean-Paul Calderone]

On Tue, Dec 6, 2016 at 12:54 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Dec 6, 2016, at 5:24 AM, Jean-Paul Calderone <exar...@twistedmatrix.com>
> wrote:
>
> Is the braid trac service also fixed - or does the next person who tries
> to update the deployment need to watch out for this same failure mode?
>
>
> The referenced commit to braid switches from multiple .install calls (i.e.
> "pip" runs) with pinned versions of things to a single one where all
> versions are pinned; as I understand it this should pin it properly for the
> future.  (It was upgrading trac in some of the things-that-depend-on-trac
> .install() calls.)
>

Cool.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

On Tue, Dec 6, 2016 at 1:08 AM, Glyph Lefkowitz 
wrote:

>
> Now I can't create tickets at all.  I hope this is related and not just
> general decay of our Trac instance :).
>
> The web UI tells me "Warning: The action "None" is not available" without
> actually creating the ticket.  Nothing that I can see appears in the logs.
> Hopefully this warning message means something to someone?
>

Not to me, unfortunately. :(


>
> -glyph
>
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

On Sat, Dec 3, 2016 at 9:28 AM, Jean-Paul Calderone <
exar...@twistedmatrix.com> wrote:

> On Fri, Dec 2, 2016 at 9:28 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>> > On Dec 2, 2016, at 4:49 PM, Jean-Paul Calderone <
>> exar...@twistedmatrix.com> wrote:
>> >
>> > I deployed this to production.  I don't see a good way to test it
>> without screwing around with twisted trunk ... so don't plan to.  I'll keep
>> an eye on real merges folks are working on and see if it's behaving as
>> desired.
>>
>> Thanks, JP!
>>
>> Would you mind updating https://twistedmatrix.com/trac
>> /wiki/ReviewProcess#Revertingachange and https://twistedmatrix.com/trac
>> /wiki/ReviewProcess#Authors:Howtomergethechangetotrunk to reflect the
>> new syntax that should be used on future merges?
>>
>>
> Updated.  As far as the behavior, do https://github.com/twisted/
> twisted/pull/614 / http://twistedmatrix.com/trac/ticket/8934 look right?
> The trac ticket is fixed but with no comment.  OTOH, I can't find the
> commit message for the PR merge so maybe that's as it should be?
>
>

Updated again.  Craig confirmed that it indeed wasn't working (at all).

Also, the site's now running trac 1.2 (was 1.0.12).  The fab target for
re-deploying trac just picks the newest release at the moment, despite the
*appearance* of being pinned to a particular version.  I don't think I'm
going to try to bite off fixing this.  The problem, though, is multiple
invocations of `pip -U` - a later instance of which is totally happy to
upgrade the trac 1.0.12 install to 1.2.

Jean-Paul



> -glyph
>>
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] memory leaks

[Jean-Paul Calderone]

On Sun, Dec 4, 2016 at 12:50 AM, Glyph Lefkowitz 
wrote:

> Following up on a Stack Overflow question from some time ago,
> http://stackoverflow.com/questions/40604545/twisted-
> using-connectprotocol-to-connect-endpoint-cause-memory-leak?noredirect=1#
> comment68573508_40604545 since the submitter added a minimal reproducer,
> I used Heappy https://pypi.org/project/guppy/ to look at memory sizing,
> and seeing large numbers of Logger instances and type objects leaking when
> using client endpoints.  It was not immediately obvious to me where the
> leak is occurring though, as I was careful to clean up the Deferred results
> and not leave them in a failure state.
>
> I am hoping that I can entice someone else to diagnose this far enough to
> actually file a bug :-).
>
>
Answered.  I didn't file a bug, I'll let someone else with ideas about
twisted.logger think about what exactly the bug is.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

On Fri, Dec 2, 2016 at 9:28 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

> > On Dec 2, 2016, at 4:49 PM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
> >
> > I deployed this to production.  I don't see a good way to test it
> without screwing around with twisted trunk ... so don't plan to.  I'll keep
> an eye on real merges folks are working on and see if it's behaving as
> desired.
>
> Thanks, JP!
>
> Would you mind updating https://twistedmatrix.com/trac/wiki/ReviewProcess#
> Revertingachange and https://twistedmatrix.com/trac/wiki/ReviewProcess#
> Authors:Howtomergethechangetotrunk to reflect the new syntax that should
> be used on future merges?
>
>
Updated.  As far as the behavior, do
https://github.com/twisted/twisted/pull/614 /
http://twistedmatrix.com/trac/ticket/8934 look right? The trac ticket is
fixed but with no comment.  OTOH, I can't find the commit message for the
PR merge so maybe that's as it should be?


> -glyph
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

I deployed this to production.  I don't see a good way to test it without
screwing around with twisted trunk ... so don't plan to.  I'll keep an eye
on real merges folks are working on and see if it's behaving as desired.

On Thu, Dec 1, 2016 at 8:50 PM, Jean-Paul Calderone <
exar...@twistedmatrix.com> wrote:

> On Thu, Dec 1, 2016 at 3:33 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>>
>>
>> Please be vocal about any roadblocks you hit.  The ops situation has
>> improved a ton since the last time you looked, but (accordingly) it's also
>> changed almost completely.
>>
>> Good luck - and hopefully you'll need a lot less of it than previously
>> ;-).
>>
>>
> Here's a new plugin that has the new behavior, maybe:
> https://github.com/twisted-infra/twisted-trac-plugins/pull/4
> thijs did a review so maybe it's all set to merge.
> There's also a config change: https://github.com/
> twisted-infra/braid/pull/226
>
> I had intended to test it with the local development environment but it
> looks like it requires downloading gigs of data for vagrant boxes and to
> upgrade a super-old ubuntu image.
> Maybe someone else could do that step.
>
> If not, maybe I'll test it on the production machine in a couple days. :)
>
> Jean-Paul
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

On Thu, Dec 1, 2016 at 3:33 PM, Glyph Lefkowitz 
wrote:

>
>
> Please be vocal about any roadblocks you hit.  The ops situation has
> improved a ton since the last time you looked, but (accordingly) it's also
> changed almost completely.
>
> Good luck - and hopefully you'll need a lot less of it than previously ;-).
>
>
Here's a new plugin that has the new behavior, maybe:
https://github.com/twisted-infra/twisted-trac-plugins/pull/4
thijs did a review so maybe it's all set to merge.
There's also a config change:
https://github.com/twisted-infra/braid/pull/226

I had intended to test it with the local development environment but it
looks like it requires downloading gigs of data for vagrant boxes and to
upgrade a super-old ubuntu image.
Maybe someone else could do that step.

If not, maybe I'll test it on the production machine in a couple days. :)

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

On Thu, Dec 1, 2016 at 3:13 PM, Jean-Paul Calderone <
exar...@twistedmatrix.com> wrote:

> On Thu, Dec 1, 2016 at 2:14 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>>
>> On Dec 1, 2016, at 10:51 AM, Jean-Paul Calderone <
>> exar...@twistedmatrix.com> wrote:
>>
>> Hi,
>>
>> In the last couple days I've noticed that there are a bunch of spurious
>> changes being made to tickets in the issue tracker.  These come from commit
>> messages that reference a GitHub PR that happens to match a ticket number
>> in trac.
>>
>> For example, https://twistedmatrix.com/trac/ticket/600#comment:11
>>
>> I guess this doesn't really hurt anything ... except it's dumping a
>> constant low level of garbage into the issue tracker and generating some
>> annoying emails (that end up having nothing to do with what the subject
>> suggests).
>>
>>
>> This is, unfortunately, going to keep happening more frequently as the PR
>> numbers get higher and the corresponding Trac tickets get less sparse.
>>
>> The way I'd *like* to address it is to change the format of our commit
>> message to namespace Trac tickets differently; instead of just "#", using a
>> URL, like "Fixes https://tm.tl/1234;.  I wouldn't even mind if we just
>> had to use the Trac wiki syntax for this, i.e. "Fixes [ticket:1234]" as
>> long as we could turn off the "#" syntax which Github also uses.
>>
>> However, this involves surgery within Trac's code, and for me personally,
>> the work required to find the relevant regexes and modify them is worse
>> than continuing to deal with the annoyance.  However, I would very much
>> appreciate it if someone else would take this on :-).
>>
>
> Where's the source for Twisted's trac deployment?  Is it actually possible
> to deploy modifications?  I'll take a look, if so.
>
>
https://trac.edgewall.org/browser/trunk/tracopt/ticket/commit_updater.py#L140
looks like the relevant regexp.  Removing the "#" branch would probably
produce the desired behavior.

However, also see
https://trac.edgewall.org/wiki/CommitTicketUpdater#Configuration.  I wonder
if commit_ticket_update_envelope would also fix it?  I don't think so
because I think these are just "references" not "commands", but I don't
know for sure.

Jean-Paul


> Jean-Paul
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

On Thu, Dec 1, 2016 at 2:14 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Dec 1, 2016, at 10:51 AM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
>
> Hi,
>
> In the last couple days I've noticed that there are a bunch of spurious
> changes being made to tickets in the issue tracker.  These come from commit
> messages that reference a GitHub PR that happens to match a ticket number
> in trac.
>
> For example, https://twistedmatrix.com/trac/ticket/600#comment:11
>
> I guess this doesn't really hurt anything ... except it's dumping a
> constant low level of garbage into the issue tracker and generating some
> annoying emails (that end up having nothing to do with what the subject
> suggests).
>
>
> This is, unfortunately, going to keep happening more frequently as the PR
> numbers get higher and the corresponding Trac tickets get less sparse.
>
> The way I'd *like* to address it is to change the format of our commit
> message to namespace Trac tickets differently; instead of just "#", using a
> URL, like "Fixes https://tm.tl/1234;.  I wouldn't even mind if we just
> had to use the Trac wiki syntax for this, i.e. "Fixes [ticket:1234]" as
> long as we could turn off the "#" syntax which Github also uses.
>
> However, this involves surgery within Trac's code, and for me personally,
> the work required to find the relevant regexes and modify them is worse
> than continuing to deal with the annoyance.  However, I would very much
> appreciate it if someone else would take this on :-).
>

Where's the source for Twisted's trac deployment?  Is it actually possible
to deploy modifications?  I'll take a look, if so.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Spurious commit/ticket links

[Jean-Paul Calderone]

Hi,

In the last couple days I've noticed that there are a bunch of spurious
changes being made to tickets in the issue tracker.  These come from commit
messages that reference a GitHub PR that happens to match a ticket number
in trac.

For example, https://twistedmatrix.com/trac/ticket/600#comment:11

I guess this doesn't really hurt anything ... except it's dumping a
constant low level of garbage into the issue tracker and generating some
annoying emails (that end up having nothing to do with what the subject
suggests).

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Changeset links in trac

[Jean-Paul Calderone]

Hi,

It looks like older links to changesets in trac are now broken.  For
example, https://twistedmatrix.com/trac/ticket/5236#comment:5 used to link
to r33400 but the link is now just plain text.

Is there a reason these links can't be restored?  It was very convenient to
have them for navigating and understanding the history of the codebase.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Some things I've learned: safer callbacks, better t.p.context

[Jean-Paul Calderone]

It's also possible to write a little bit of testing library code and get
both versions of the test nearly for free.  Unfortunately, I very rarely
see test suites written that way.  I think many people don't realize there
are two cases to handle or believe testing both cases is too expensive for
them (perhaps because they don't realize you can reuse the code for doing
so to save on the cost).

Jean-Paul

On Wed, Oct 19, 2016 at 5:54 PM, Glyph Lefkowitz 
wrote:

>
> On Oct 19, 2016, at 2:47 PM, Itamar Turner-Trauring 
> wrote:
>
> On Wed, Oct 19, 2016, at 05:45 PM, Itamar Turner-Trauring wrote:
>
> Well... I had a test that went through synchronous Deferred path. And
> yeah, it was easier to write than async test. But it failed to catch a bug
> that was only in async case. So the problem I see is that supporting both
> in Deferred means you need twice the number of tests each time you use
> Deferreds.
>
>
> Er, that was unclear. I had a bug that wasn't caught by tests because it
> passed with sync Deferred and failed with async Deferred callback, and I
> didn't have tests for latter.
>
>
> To be clear: I do see this as a *downside* to Deferred's architecture;
> it's a tradeoff.  I see the loose coupling with the event loop as a
> worthwhile upside.
>
> However, it's totally possible to write the async Deferred callback case
> as well, just by having the test fire the Deferred after returning to the
> test body instead of firing it before :-).
>
> -glyph
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] How do you determine the buffer size of a transport - a use-case for not using back pressure

[Jean-Paul Calderone]

There are at least two buffers (per direction) you might be interested in.
You can get the kernel buffer size with getsockopt - SO_SNDBUF and
SO_RCVBUF.  Then, there may also be a user-space buffer (or perhaps more
than one) managed by the transport implementation.  The details of this are
entirely up to the transport so there's no general answer apart from "read
the implementation, contribute a patch implementing a public interface for
retrieving the information".

Jean-Paul

On Fri, Aug 19, 2016 at 10:08 AM, Steve Morin  wrote:

> Anyone know how do you determine the buffer size of a transport, to know
> how much data is waiting to be transmitted from using transport.write?
>
> Or how you would go about adding that ability to a reactor/transport?
>
> On Wed, Aug 17, 2016 at 3:43 PM, Steve Morin 
> wrote:
>
>> Twisted Community
>>
>> Problem: How do you determine the buffer size of a transport, to know how
>> much data is waiting to be transmitted from using transport.write?
>>
>> Wait! You're going to say: use the Producer Consumer API (
>> http://twistedmatrix.com/documents/current/core/howto/producers.html )
>>
>> To do what: So that instead of using back pressure I can check the buffer
>> and when it's "too big/full" can decide to do something to the transport I
>> am writing to:
>>
>> Slow transport handling options:
>>
>> - Buffer to disk instead of memory
>> - Kill the transport
>> - Decide to skip sending some data
>> - Send an error or message to the transport I am writing to
>> - Reduce the resolution, increase the compression (things like video or
>> audio)
>>
>> Why not use back pressure?: Some use-cases and some protocols this
>> doesn't make sense.
>>
>> Examples:
>> - You're sending video and if the receiver can't keep up you want to
>> downgrade or upgrade the quality of the video, but if you don't know if you
>> can't tell how much it buffering.
>> - You're receiving from one connection and then broadcasting what you
>> received to multiple clients and you want to handle it by sending an error
>> to a client that doesn't keep up
>> - You're sending a real-time protocol and want to skip sending some data
>> that's no longer relevant if the buffer is too slow.
>>
>> On a server what are the consequences:
>> Too much buffering in many transport write buffer can cause a server to
>> fail.  I don't know how to keep track of this to proactively without access
>> to the buffer sizes.  Resolutions can then be to, not accept new
>> connections when memory pressure is too high, kill connections with the
>> weakest/slowest clients or have a protocol that can have client switch
>> connections to new servers when instructed to spread out the load.
>> 1) I would like to hear on how people would solve this sort of problem in
>> Twisted for a server?
>> 2) I would like to hear people opinions on this in general.
>>
>> Tobias Oberstein - BCC'ed you on this email because you seems to have
>> tackled similar problems (based on the mailing list) and would really love
>> to get your take on this too.
>>
>> Glyph and Jean-Paul, you're also big on those threads so any opinions you
>> have would be appreciated as well.
>>
>> Some of my background research
>>
>> * http://twistedmatrix.com/pipermail/twisted-python/2015-Janua
>> ry/029064.html
>> * Later but good in the chain: http://twistedmatrix.com/piper
>> mail/twisted-python/2015-January/029071.html
>>   * Twisted receiving buffers swamped?
>>   * Summary: Great thread but runs into a tangent for a while and picks
>> up good at the end again discussing producer again and the need for
>> backpressure
>>   * Other:
>> * http://crossbar.io/
>> * http://tavendo.com/
>>   * Scenario: "Twisted is reading off the TCP stack from the kernel and
>> buffering in userspace faster than the echo server is pushing out stuff to
>> the TCP stack into the kernel. Hence, no TCP backpressure results, netperf
>> happily sends more and more, and the memory of the Twisted process runs
>> away."
>>   * Confirmed: Data isn't Buffered in "userspace inside Twisted, and
>> before data is handled by the app in dataReceived."
>> * https://twistedmatrix.com/pipermail/twisted-python/2010-Sept
>> ember/022900.html
>>   * How to cap the buffering size of data to be sent in Protocol class
>>   * Summary: Same issue, very short no good info
>> * https://twistedmatrix.com/pipermail/twisted-python/2012-Marc
>> h/025416.html
>>   * Limit on transport.write
>>   * Summary: Similar issue, very short no good info, glyph confirms that
>> transport.write buffers everything sent to it.
>> * https://twistedmatrix.com/pipermail/twisted-python/2012-Febr
>> uary/025215.html
>>   * pushing out same message on 100k TCPs
>>   * Summary: Interesting discussion but different issue - interesting
>> aside about irc spanning trees for a large broadcasts
>> * http://twistedmatrix.com/pipermail/twisted-python/2008-April
>> /017503.html
>>   * Question on push/pull 

Re: [Twisted-Python] Waiting time for tests running on Travis CI and Buildbot

[Jean-Paul Calderone]

On Mon, Aug 15, 2016 at 7:03 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Aug 15, 2016, at 2:11 PM, Jean-Paul Calderone <
> exar...@twistedmatrix.com> wrote:
>
> On Mon, Aug 15, 2016 at 4:38 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
> There's a lot that we can do to make Travis almost that fast, with
> pre-built Docker images and cached dependencies.  We haven't done much in
> the way of aggressive optimization yet.  As recently discussed we're still
> doing twice as many builds as we need to just because we've misconfigured
> branch / push builds :).
>
>
> Hm... pre-built dockers also takes effort to keep them updated... and
> then we will have a KVM VM starting inside a docker in which we run
> the tests...
>
> ...and we would not be able to test the inotify part.
>
>
> Not true:
>>
>>
>>1. We can have one non-containerized builder (sudo: true) for testing
>>inotify; no need for KVM-in-Docker (also you can't do that without a
>>privileged container, so, good thing)
>>
>> I'm curious about the details of how such a configuration would work.
> Since there is only one travis configuration per repository (well, per
> branch, technically, don't think that makes a difference here) and the sudo
> configuration is global (isn't it?), I always though either a project had
> to pick sudo or not sudo and you couldn't have a mix of builds with each.
>
>
> I had just assumed that it would be per-matrix-entry, and while it looks
> like I'm correct, it's much less obvious than I had thought.  If you look
> at the second example code block under this heading:
>
> https://docs.travis-ci.com/user/multi-os/#Example-Multi-OS-Build-Matrix
>
> you'll see that one of the matrix/include entries has a 'sudo: required'
> tag, which means "non-container-based" please.  Presumably you can mix
> those.
>

Ah, neat.  I hope that works (and wish I'd noticed that example a couple
months ago).

Jean-Paul


>
> The docs on "migrating to the container-based infrastructure" do make it
> sound like this is impossible though, and this isn't nearly as clear as I'd
> like, so it would be nice to actually experiment and see what happens...
>
> (Wonky quoting thanks to gmail's wonky web interface, sorry)
>
>
> It wasn't too bad - certainly well-worth suffering through for your
> contribution :).
>
> -glyph
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Waiting time for tests running on Travis CI and Buildbot

[Jean-Paul Calderone]

On Mon, Aug 15, 2016 at 4:38 PM, Glyph Lefkowitz 
wrote:

There's a lot that we can do to make Travis almost that fast, with
pre-built Docker images and cached dependencies.  We haven't done much in
the way of aggressive optimization yet.  As recently discussed we're still
doing twice as many builds as we need to just because we've misconfigured
branch / push builds :).


Hm... pre-built dockers also takes effort to keep them updated... and
then we will have a KVM VM starting inside a docker in which we run
the tests...

...and we would not be able to test the inotify part.


Not true:
>
>
>1. We can have one non-containerized builder (sudo: true) for testing
>inotify; no need for KVM-in-Docker (also you can't do that without a
>privileged container, so, good thing)
>
> I'm curious about the details of how such a configuration would work.
Since there is only one travis configuration per repository (well, per
branch, technically, don't think that makes a difference here) and the sudo
configuration is global (isn't it?), I always though either a project had
to pick sudo or not sudo and you couldn't have a mix of builds with each.

(Wonky quoting thanks to gmail's wonky web interface, sorry)

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Waiting time for tests running on Travis CI and Buildbot

[Jean-Paul Calderone]

On Sun, Aug 14, 2016 at 7:10 PM, Glyph Lefkowitz 
wrote:

>
> > On Aug 14, 2016, at 3:38 AM, Adi Roiban  wrote:
> >
> > If you think that we can raise $6000 per year for sponsoring our
> > Travis-CI and that is worth increasing the queue size I can follow up
> > with Travis-CI.
>
> I think that this is definitely worth doing.
>

How might we guess this would compare to $6000/year development effort
spent towards  speeding up the test suite?

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Coverage exceptions

[Jean-Paul Calderone]

On Thu, Jun 30, 2016 at 6:25 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Jun 30, 2016, at 04:13, Jean-Paul Calderone <exar...@twistedmatrix.com>
> wrote:
>
> On Thu, Jun 30, 2016 at 6:43 AM, Adi Roiban <a...@roiban.ro> wrote:
>
>> Hi,
>>
>> Recently we have introduced a hard check of 100% coverage for all changes.
>> This is done via coverage + codecov + github protected branches.
>>
>> Now, if your patch is not 100% covered github will not let you merge it.
>>
>> See for example this change:
>> https://github.com/twisted/twisted/pull/261/files#diff-0fea8a8ca713deb7ea6a10053273319aR2360
>>
>> The errback is there to help with test failures ... but the test should
>> never fail, so that errback is never called... and that line is not covered.
>>
>>
> It doesn't always make sense to require 100% execution of all test code.
> It's not at all uncommon to only have code in a test suite that runs when a
> test fails.  Historically, Twisted has never had a requirement of 100%
> execution of test code.  The only test suite coverage requirements that
> have commonly been requested or enforced is for coverage of implementation
> code.
>
> I'd suggest removing the coverage enforcement for test suite code.
>
>
> I am inclined to disagree, albeit mildly.
>
> When one is writing a deliberately un-covered path in test code,
> presumably, one is writing either a test helper - a mock, fake, or utility
> for setting up a real implementation - or an assertion method.
> Historically, I believe that when we've invested more heavily in making
> these utilities "real" themselves, and not just throwaway stuff inline in a
> test method or module, the benefits have far outweighed the costs.  In fact
> the availability of proto_helpers is one of the selling points of Twisted
> as opposed to other competing engines.
>

I mostly agree with this.  However, I was thinking of a slightly different
pattern when I wrote my earlier email.  Here's are a couple (fictional)
examples of that pattern one might find in unit tests for application code
(and there's nothing Twisted-specific here):

if foo:
self.fail("Foo!")

try:
foo()
except:
bar
else:
self.fail("Foo :(")

It's not exactly that this *can't* be code that's executed in a passing run
of the test suite.  It's more a question of what the right balance point
is.  If someone wants to generalize logic like this (and, fortunately,
someone did generalize these *particular* examples - they're assertFalse
and assertRaises, respectively) then that's great and the result is a
higher level of confidence resulting from a successful run of the test
suite.  I'd suggest that if tests like these exercise all of the
implementation code (on a successful run), though, then you've still
achieved a pretty high level of test coverage and maybe further efforts are
more productively directed elsewhere (increasing the coverage level of
other implementation code in Twisted, for example :).

If folks want a higher bar than this, I'm not going to argue (at least not
much, at least not now).  The bar *hasn't* been this high in the past
though (and there are many many such cases to be found in Twisted's test
suite right now and I don't have the impression this has ever been *much*
of a source of problems).


> Therefore, I think that asking folks to add independent test coverage to
> verify their fakes and ensure that the failure-reporting of their assertion
> messages are helpful in the event a test fails is a generally good idea,
> and we should keep the requirement for 100% coverage on both test and
> implementation coverage.
>
> However, if there is contention around this, I'd much rather get a ratchet
> in place for implementation code that's reliable and everyone is happy
> with, so I'm OK with disabling coverage reporting for our *.test.* packages
> as a step towards that.
>
>
I completely agree that fakes should be verified.  So much so that I'm not
even sure I believe in fakes in *general* anymore.  Instead, you should
just have easy to use interfaces and ship inexpensive implementations
alongside whatever other implementations you also need.  And all those
implementations should have great test coverage.  I also completely agree
that when tests fail, they should do so in a meaningful way.  I suspect
slightly the implication that automated test coverage for the failure case
demonstrates the failure is reported meaningfully, though. :)  I think
we're still stuck with relying on humans (authors, reviewers) to verify
that property.

Jean-Paul


> How should we proceed with these changes?
>>
>> Maybe this is not the best example and that code could be refactored...
>> but I think that the topic of ignoring mi

Re: [Twisted-Python] Coverage exceptions

[Jean-Paul Calderone]

On Thu, Jun 30, 2016 at 6:43 AM, Adi Roiban  wrote:

> Hi,
>
> Recently we have introduced a hard check of 100% coverage for all changes.
> This is done via coverage + codecov + github protected branches.
>
> Now, if your patch is not 100% covered github will not let you merge it.
>
> See for example this change:
> https://github.com/twisted/twisted/pull/261/files#diff-0fea8a8ca713deb7ea6a10053273319aR2360
>
> The errback is there to help with test failures ... but the test should
> never fail, so that errback is never called... and that line is not covered.
>
>
It doesn't always make sense to require 100% execution of all test code.
It's not at all uncommon to only have code in a test suite that runs when a
test fails.  Historically, Twisted has never had a requirement of 100%
execution of test code.  The only test suite coverage requirements that
have commonly been requested or enforced is for coverage of implementation
code.

I'd suggest removing the coverage enforcement for test suite code.


> How should we proceed with these changes?
>
> Maybe this is not the best example and that code could be refactored...
> but I think that the topic of ignoring missing coverage is still valid.
>
> I suggest to introduce `  # pragma: no cover`
>
> and update the coverage config with
>
> [report]
> exclude_lines =
> pragma: no cover
>
>
This seems like the wrong solution to me.  It forces contributors to do
extra work to mark their test code as an exception *and* provides a
mechanism for incorrectly bypassing the check by using a no-cover pragma in
implementation code.

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted listening on UDP port, why?

[Jean-Paul Calderone]

Hi,

What do the logs for the app say?  Twisted logs a message when it binds a
UDP port.

Or, another though, you could put a breakpoint on listenUDP (or socket.bind
or something) and then run the process under pdb and look at the stack
trace.

You could also try sending some traffic to the port and see what happens.
:)  Maybe you'll get something back that identifies it or maybe you'll
provoke some more logging code somewhere.

Jean-Paul

On Wed, Apr 20, 2016 at 7:33 AM, Phil Mayers 
wrote:

> On 20/04/16 01:01, Glyph wrote:
>
> Perhaps this is libc's DNS client?  Twisted doesn't do anything like this.
>>
>
> It does something similar with win32reactor IIRC?
>
>
> http://twistedmatrix.com/trac/browser/tags/releases/twisted-16.1.1/twisted/internet/posixbase.py#L60
>
> ...but that binds to 127.0.0.1 not 0.0.0.0
>
> glibc doesn't hold it's DNS sockets open AFAIK - it closes them once the
> reply is done.
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Questions about adding documentation

[Jean-Paul Calderone]

On Fri, 31 Jul 2009 11:55:08 -0500, Kevin Horn kevin.h...@gmail.com wrote:
 [snip]

I'd love to see a documentation reboot using Sphinx, but not if it's going
to be a half-baked, never-finished project.

 [snip]

I'd also be interested in hearing the opinions of some of the core Twisted
guys on the various things we've been talking about here.  What do you guys
think about using a different docs system than what is being used now?  If
you guys are all dead set against it, there's not much point hashing out the
details...

I don't find that people trying to use Twisted complain about the
presentation of the documentation.  I find that they complain about
its content.

So, I think that it's the content that needs to be addressed.  I don't
*think* that switching to Sphinx (or anything else) is going to make any
different to the content of the current documentation.  One might argue
that Lore is a significant barrier to entry for new contributions to the
documentation, but you'd have to try pretty hard to convince me.  Pretty
much anyone can write some simple (Lore x)html.  And if they can't, then
there are other people willing to translate plain text into Lore input
documents.

Also, what do the Twisted core devs think about having a secondary
wiki/cookbook thingy outside of the core docs?

As a staging area for development of future core docs, I think I would
recommend using a version control system (perhaps a distributed one),
not a wiki.

As something intended to be user-facing, I don't think it's a great idea.
Of course, there *is* a wiki hosted on the website already...  And it has
some documentation on it...  So what's being proposed, exactly? :)

Obviously I'm speaking for myself, not for all the other people who have
contributed to Twisted.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] License for code in twisted's sandbox?

[Jean-Paul Calderone]

On Mon, 27 Jul 2009 20:41:42 -0400, Glyph Lefkowitz gl...@twistedmatrix.com 
wrote:
On Mon, Jul 27, 2009 at 10:34 AM, Vishal Shetye vishalshe...@gmail.comwrote:

 I had posted a query about laxdb on this list
 http://twistedmatrix.com/pipermail/twisted-python/2009-July/020030.html
 laxdb code is in sandbox and is not released with twisted. The file itself
 does not have any license.
 Can it be used assuming the same license as that of twisted.


In general you can assume code in Twisted's sandbox is released under the
same license as Twisted itself.

You can assume it, but it'd be safer to read the license file :)

http://twistedmatrix.com/trac/browser/sandbox/LICENSE

It's safest to check with the author personally (in this case, mg) in case
there is some reason for an exception.

So you definitely should do this.


However, I would strongly discourage you from using this code in any
capacity other than trying to contribute patches which add it (and by it I
mean tested, working versions of it) to some existing twisted module, in
this case adbapi.  As it says, it is experimental, untested, incomplete,
it hasn't been updated in 4 years, and I assume it is totally unsupported,
both by its author and by the Twisted team in general.  If it breaks, you
can't even keep both pieces, you have to dispose of them as toxic waste.


That's all still excellent advice.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Incorrect Another twistd server is running...

[Jean-Paul Calderone]

On Wed, 22 Jul 2009 00:41:20 +0200, Simone Deponti shywolf9...@gmail.com 
wrote:
On 07/21/2009 07:10 PM, Christopher Armstrong wrote:
It's not necessarily a twistd process. It could be any process with
the same pid. checkPID only checks to see if the pid exists, not that
it's the PID of a twistd process.

Disclaimer is, I might be extremely naive here. I have to admit I wasn't 
able to figure out when checkPID is called (before or after a possible 
daemonization? Since with how you have to write daemons in UNIX, doing a 
os.getpid() _before_ daemonizing is not going to work the way I intend it 
to).

Anyway, what is currently done is: os.kill is invoked with signal 0, if we 
get no OSError then sys.exit is invoked.

What I propose is if no OSError is raised, we first check that os.getpid() 
is different from the pid we read from the file, and if they are equal we 
remove the stale file.

Patch against r27172 attached.

Don't shout at me if I haven't got it (it honestly seems way too easy: there 
must be the trick ;))



This will fix the problem in one very unlikely case, but leave it broken in
the other somewhat more likely cases.  The problem is that some *other*
process may have been started on your system and given the PID from the
pidfile.  Your proposed change won't fix that case.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Problem with twisted.names.client.lookupPointer

[Jean-Paul Calderone]

On Sun, 19 Jul 2009 06:34:21 +0100, Kieran Simkin kie...@digital-crocus.com 
wrote:
Hi there,
I'm trying to do some reverse DNS lookups with twisted and I don't seem to 
be able to get even the most simple of test cases to work. I've tried it on 
a couple of different installations with the same results, perhaps someone 
can help me?

lookupPointer is a pretty low-level interface.  It's just going to give you
back the records that the DNS server it talks to gives back.  This also means
that its behavior can very based on which DNS servers it ends up using.  You
are probably interested in a higher level API that does the additional
sorting-out that's required to get the value you're actually asking for.
Unfortunately, this API isn't provided by Twisted Names (yet! :).

That said, you might be encountering a problem not related to this.

My test case is this:
--
from twisted.names import client
from twisted.internet import reactor
def callback(a):
print repr(a)

client.lookupPointer('128.244.209.91.in-addr.arpa').addCallback(callback)
reactor.run()
--

The output I'm getting from this program is as follows:
([RR name=128.244.209.91.in-addr.arpa type=PTR class=IN ttl=9777s 
auth=False], [RR name=244.209.91.in-addr.arpa type=NS class=IN ttl=8348s 
auth=False, RR name=244.209.91.in-addr.arpa type=NS class=IN ttl=8348s 
auth=False], [RR name=ns1.solsticehosting.net type=A class=IN ttl=8348s 
auth=False, RR name=ns2.solsticehosting.net type=A class=IN ttl=8348s 
auth=False])

Nowhere in this output is the actual PTR record - the first entry looks like 
it should be the PTR record, but the name is '128.244.209.91.in-addr.arpa' - 
I'm expecting it to be node2.digital-crocus.com. Can anyone clarify whether 
I'm doing something wrong or if this is in fact a bug?

What you get back from the lookup methods are RR objects.  These are the
generic wrapping layer around any record-specific data.  If you look at the
first entry's payload attribute, I think you'll find a Record_PTR instance
which has the information you're after.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Cancel/stop Deferred() at any time

[Jean-Paul Calderone]

On Tue, 14 Jul 2009 11:10:10 -0400, vit...@synapticvision.com wrote:
Hello,

is there any way to cancel the Deferred callback in those examples?

1.
d = defer.Deferred()
d.addCallback(methodA).addErrback(methodErr)
d.callback(None)

2.
d = methodB().addErrback(methodErr)
wfd = defer.waitForDeferred(d)



Appreciate any help.


Deferred doesn't have any cancellation features.  You can generally achieve
some sort of cancellation by supporting this in the application code creating
and consuming the Deferreds involved.

The details of how to do this depend on what exactly you want to happen when
you cancel the operation.

You can find one example of this in the implementation of Twisted's internal
thread-based name resolver:

http://twistedmatrix.com/trac/browser/trunk/twisted/internet/base.py#L193

You can find other examples in the implementation of any other Deferred-
returning API which also supports timeouts, eg the web client:

http://twistedmatrix.com/trac/browser/trunk/twisted/web/client.py#L198

Hope this helps,

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] imap and noop

[Jean-Paul Calderone]

On Wed, 08 Jul 2009 15:33:49 +0100, Darren Syzling dsyzl...@gmail.com wrote:
I'm new to twisted and was experimenting trying to create an imap server
based on the example within Twisted Network Programming Essentials. My
test client is Thunderbird.

I'm wondering if anyone has had any experience handling the 'noop'
command here's my scenario:
- I have two unread messages in a folder, I open thunderbird, select
this folder and then select trash - trash has no messages.
- Each folder is selected and fetched with a uid:*
- My server correctly responds.
- Each time I now select the trash folder since it didn't have any
messages to start with Thunderbird continues to send noops. I think it's
waiting for my server to respond with exists or recent.
- Ok so I continue, I read and delete the two messages in my folder,
Thunderbird issues copy commands, I dutifully flag the messages in the
folder as \Deleted and the messages are added (via addMessage) to the
Trash folder with the correct flags - \Seen.
- But now if I select the Trash folder thunderbird just sends noop -
again it looks like it wants the server to send back exists or recent to
indicate there are now messages in that folder - even though it's just
copied stuff to the folder.

If in the above scenario I don't select the trash folder first and
messages are deleted everything works fine - but in that case the client
is sending fetch uid:*. It seems that if Thunderbird detects no messages
in a folder it will noop until the server responds with an update.

Anyone seen or worked around this? Are there hooks in the imap framework
to allow a mailbox or my mailserver to respond to noops? I don't appear
to have any mailbox listeners subscribed and I tried overriding the noop
for my derived IMAP4Server protocol derived class - but to no avail.


You may want to use the IMailboxListener implementation on IMAP4Server to
handle this case (IMAP4's complicated, I'm not sure this is the right
direction to pursue, but it's *a* direction :).

The modeChanged, flagsChanged, and newMessages methods on IMAP4Server are
intended for use by your mailbox implementation.  In this case, it sounds
like you want to have your trash mailbox call newMessages when the messages
are copied into it.  This requires a bit of legwork on your part, keeping
track of the IMAP4Server protocol (or possibly more than one!) from the
mailbox code, but the result should be that when Thunderbird sends those
NOOPs, the server responds with one or both of EXISTS or RECENT (untagged).

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Failed to load application: cannot import name app

[Jean-Paul Calderone]

On Tue, 7 Jul 2009 17:20:41 +0200, Jesper Taxbøl jes...@taxboel.dk wrote:
Hi Guys,

I stumpled upon a small dns filter program, that I am having trouble
running, due to twisted.internet.app being deprecated. I get the following
output Failed to load application: cannot import name app

Im running twistd (the Twisted daemon) 2.5.0

Any advice on how to proceed?

Kind regards

Tax

 [snip]
application = app.Application('Non caching anti-verisign domain name
resolver')
application.listenUDP(53, p)
application.listenTCP(53, f)


I think the `listenUDP´ line probably causes an AttributeError to be raised.
This prevents the tac file from being evaluated completely, so twistd ends
up with an exception instead of an application.  Take a look at the classes
in twisted.application.internet (eg UDPServer, TCPServer) for the way to set
up servers the new way.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] sudo asking for a password during utils.getProcessOutputAndValue()

[Jean-Paul Calderone]

On Tue, 07 Jul 2009 15:06:49 -0700, Steve Freitas sfl...@ihonk.com wrote:
Hi Maarten,

On Tue, 2009-07-07 at 23:46 +0200, Maarten ter Huurne wrote:

 Did you try args=('blah', 'blah')?


Ah ha! That did it. Wonder why that works.


I suppose your sudoers configuration file wasn't recognizing the command
before (since it was indeed a different command) and so wasn't letting you
go through the no-password case.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] (Mis)indented Posts (list administrativia)

[Jean-Paul Calderone]

On Fri, 3 Jul 2009 10:32:07 -0400, Gerrat Rickert grick...@coldstorage.com 
wrote:
...ok, I've created a new post twice on this list twice, and both times
my post has been indented and placed under a completely
different/irrelevant post

Here:
http://twistedmatrix.com/pipermail/twisted-python/2009-June/019859.html
And Here:
http://twistedmatrix.com/pipermail/twisted-python/2009-July/019902.html

...if this post hijacks a thread as well, then it will be 3 times.

...this seems to be something specific to the twisted python mailing
list:
http://www.mail-archive.com/mailman-users%40python.org/msg53996.html


I don't really know much about how mailman generates those threaded views.
twisted-python is running on an unmodified mailman 1:2.1.9-9ubuntu1 install.
Perhaps this could be caused by exim4 configuration somehow (we do have
various exim4 configuration customizations), but that's the only thing I
can think of.  Any pointers on what information is used to determine the
threading structure of messages?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] http.Request and JSON

[Jean-Paul Calderone]

On Tue, 30 Jun 2009 13:12:05 +0100, Reza Lotun rlo...@gmail.com wrote:
Hi,

On Tue, Jun 30, 2009 at 12:56 PM, Tristan
Seligmannmithra...@mithrandi.net wrote:
 To be safer, I do a request.content.getvalue() since content is a
 cStringIO object and you can never be sure if somewhere along the
 chain of processing someone hasn't done a .read and forgot to do a
 .seek(0,0).

 You should probably do the seek/read instead, since I don't believe
 this is guaranteed to be a cStringIO; it might be an actual file on
 disk, or something else.

If that's true then there really should be a .get_body() method on the
request object.

Why can't you seek and read?

Also, this discussion would be more appropriate on the twisted-web mailing
list.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] How to disable XMLRPC logging

[Jean-Paul Calderone]

On Wed, 24 Jun 2009 16:53:58 -0500, Dustin Land revenantst...@gmail.com wrote:
Greetings!

Currently I'm running two daemons that talk to each other over xmlrpc.
One runs an implementation of internet.TCPServer with an xmlrpc.XMLRPC
handler.
The other simply uses web.xmlrpc.Proxy.

In the logs for the first I get a flood of...
2009-06-24 15:31:35-0500 [-] 127.0.0.1 - - [24/Jun/2009:20:31:34 +]
POST / HTTP/1.0 200 297 - Twisted/XMLRPClib
In the second...
2009-06-24 16:44:13-0500 [-] Starting factory
twisted.web.xmlrpc._QueryFactory instance at 0x2d9afc8
2009-06-24 16:44:13-0500 [QueryProtocol,client] Stopping factory
twisted.web.xmlrpc._QueryFactory instance at 0x2d9afc8

Is there a way to disable this?  I saw in the archives that
HTTPClientFactory.noisy can be set to False.  Is there something similar for
both of these instances?

Not really.

_QueryFactory is a subclass of ClientFactory, so like HTTPClientFactory, it
has the `noisy´ attribute which controls whether it logs these messages.  The
trouble here is that you don't have access to the _QueryFactory instance.  It
might be a nice enhancement to either add a way to get at it, or perhaps to
just give `Proxy´ a `noisy´ attribute which it copies onto the _QueryFactory
instance it creates.

It might also be nice to get rid of this `noisy´ stuff and instead have
ClientFactory (and subclasses) always log this information, then teach the
log observer (ie, the thing which actually writes these messages to a log
file) how to be more selective in what it actually reports.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] SIP Registrar

[Jean-Paul Calderone]

On Mon, 22 Jun 2009 15:23:28 +0200, Guenter Dannoritzer 
tvmrfusnu...@spammotel.com wrote:
Hi,

I am trying to run a SIP registrar.

So far I used an example that comes with the Perl Net:Sip Package:

http://search.cpan.org/~sullr/Net-SIP/

As I am more familiar with Python, but not yet with twisted, I tried to
find a simple example that runs a SIP registrar based on twisted.

I searched this mailing list and found a post about rather using the
Divmod Sine sip implementation than this one. On the other side it seems
like that some of this functionality is back ported into twisted.

The back port is not yet complete, so Divmod Sine is still more functional
and easier to use (as well as less buggy) than the SIP code in Twisted.  I
would suggest either helping to complete the port, or using Divmod Sine
directly.

You can read more about the progress of the port in the issue tracker:

  http://twistedmatrix.com/trac/ticket/3584

(including a detailed list of what remains to be done).

Does the current twisted implementation provide enough functionality to
implement a SIP registrar?

Probably, but I don't know of any examples of doing so.  The Divmod Sine
code definitely provides this functionality, and you can find a registrar
implemented in Divmod Sine itself.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Dropin.cache errors after clean install of twisted-8.2.0

[Jean-Paul Calderone]

On Wed, 17 Jun 2009 14:37:01 -0700, Alex Clemesha cleme...@gmail.com wrote:
On Wed, Jun 17, 2009 at 1:41 PM, Alex Clemeshacleme...@gmail.com wrote:
 Hi,

 Just type sudo twistd and the errors will go away.
 (You may already know of this work-around)

  I agree with you that it would be nice for these to never show up in
 the first place.
 I looked at the ticket you mention, but it seems that the cleaner solution
 would be to resolve these issues at install/upgrade time.

Just to follow up on this a bit more, reading from here:
http://twistedmatrix.com/projects/core/documentation/howto/plugin.html
in the section Plugin Caching, there is this quote:


[snip]... when installing or removing software which provides Twisted
plugins, the site administrator should be sure the cache is
regenerated. Well-behaved package managers for such software should
take this task upon themselves, since it is trivially automatable.


The way I read this, requiring the site admin to run sudo twistd to
regenerate the cache file is a feature, not a bug.
Twisted devs: is this the case?  Still maybe the install should do the
trivially automatable bit?

The package managers referred to in that document are things like dpkg
and rpm which easily support running arbitrary code after installation.

It's rather difficult to automate with distutils, though (which is how
the original poster installed Twisted, I am assuming).  However, if
someone figures out how, a patch will be accepted happily. :)

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] can addCallback accept list of deferreds?

[Jean-Paul Calderone]

On Tue, 16 Jun 2009 13:23:17 +0200, Pet petshm...@googlemail.com wrote:
Hi,

I'm trying to process several items in callback function writeLog
d.addCallback(self.writeLog,g=g)
see below.

Can writeLog return DeferredList instead of Deferred?

Yes, it can.  Deferred and DeferredList are mostly interchangeable.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] SerialPort.loseConnection() BUG

[Jean-Paul Calderone]

On Sat, 13 Jun 2009 22:26:48 +0100, James Mansion 
ja...@mansionfamily.plus.com wrote:
gl...@divmod.com wrote:
 Since I'm seeing more and more messages where someone is top-posting, I
 think it might be helpful if we sent a more specific example of a

You could try not being anal about it.  In 2 decades working in
investment banks I
don't think I've seen a work evironment where it isn't the norm.  Some
of us even
prefer it.


Let's not make a big deal out of this, please.  I'm sure we all have
better things to do.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] SerialPort.loseConnection() BUG

[Jean-Paul Calderone]

On Wed, 10 Jun 2009 11:20:15 +0800, biziap biziap fet...@gmail.com wrote:

Please don't top-post.

Maybe you can try to call transport.flushInput(), transport.flushOutput()
before loseConnection().


No, this is wrong.  Do not call these methods, ever.


2009/6/9 Nestor A. Diaz nes...@tiendalinux.com:
 Hello, there is a bug on the SerialPort Win32 implementation (don't know
 about the others), when i call the loseConnection method, it returns:

 twisted matrix exceptions.AttributeError:  'SerialPort' object has no
 attribute '_tempDataBuffer'

 if i wrap the serialport class and put a :

        self._tempDataBuffer = ''

 in the __init__ method, i get:

 raise NotImplementedError(%s does not implement writeSomeData %
 exceptions.NotImplementedError: __main__.MySerialPort does not implement
 writeSomeData

 however the port gets closed the right way.


Can you provide a minimal example which demonstrates this behavior?  It is
perhaps a bug in Twisted, but it is difficult to know without being able to
see *exactly* how the behavior is triggered.

Thanks,

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] stop conditions and deferToThread()

[Jean-Paul Calderone]

On Wed, 10 Jun 2009 12:04:16 +0200, Thomas Jakobsen 
thomas.jakob...@alexandra.dk wrote:
Hi

As discussed in a previous thread

   http://twistedmatrix.com/pipermail/twisted-python/2009-May/019717.html

a task put in its own thread via deferToThread() won't stop even
though the reactor has stopped. It has to carry out its own check that
the reactor is running and stop if that is not the case.

deferToThread is optimized for short-running tasks.  Thread creation costs
are amortized over the lifetime of the process.  This is because the intended
use of deferToThread is to run functions which compute some result and then
return, /not/ for functions which run forever (or at least until the process
is ready to exit).

You should try to find a way to re-arrange your code so that there is no
need to check if the reactor is still running - just do one piece of work
in a thread at a time.  When your non-threaded application code gets the
result back, if no one has asked it to shut down, then it can ask for
another job to be executed.

My question is: Is it possible to use anything else than the
reactor.running as stop condition in this way?

Also, I want to point out that while `reactor.running´ isn't private (since
it doesn't start with an underscore, as Twisted's privacy policy dictates it
would need to), it still isn't something you really want to be using.  It's
not part of any interface, which means there is no guarantee it will be
provided by all reactors.

It's also not really for you.  It's a flag that the reactor uses to track
its own internal state.  Its meaning may not actually correspond to the
meaning you'd like to assign to it.  Whenever you're trying to do something
like this (which I still don't recommend), you should create your own state
tracking and use that instead.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Question about a custom byte oriented TCP protocol

[Jean-Paul Calderone]

On Tue, 2 Jun 2009 16:09:00 +0100, Michael Thompson michae...@gmail.com wrote:
Hi,
   I'm trying to use twisted to talk to an existing system that uses a byte
oriented protocol stuffed inside a TCP packet. I have implemented this using
the IntNStringReceiver modified to look at some bytes to calculate the size
of the string to receive.

For simple test cases where I send a single packet this works but if I send
multiple requests they all end up in the same tcp packet which the existing
server doesn't like. How can I ensure that only a single command\string is
sent per tcp packet?

In general, this is impossible.  TCP does not work that way.

For a specific application, you may be able to achieve this by only sending
another message after you *know* the server has received an earlier message.
You can probably achieve this if the server sends you back an acknowledgment
of each message - then you wait to receive an acknowledgment before sending
another message.

However, the purpose of IntNStringReceiver - arguably the sole purpose - is
to provide a framing layer for the messages being exchanged with it.  This
framing layer exists so that if messages do get squished together (or broken
up, as may also happen) by TCP, they can still be properly interpreted.  If
the server breaks when multiple commands/strings are received by it at once,
then perhaps it is broken and needs to be fixed.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] wxreactor and win32eventreactor

[Jean-Paul Calderone]

On Tue, 2 Jun 2009 11:07:38 -0500, Ryan Nowakowski tuba...@fattuba.com wrote:
I'm trying to use twisted's SerialPort stuff in my Win32 wxPython app.
I'm using the wxreactor, however I'm getting this error:

  AttributeError: 'WxReactor' object has no attribute 'addEvent'


Indeed, as you've discovered, the serial support on Windows only works with
win32eventreactor.

When I switch to the win32eventreactor my serial port code works fine
but then of course, the GUI won't work.  Any idea how to get SerialPort
working with wxreactor on Windows?

I think this would involve porting wxreactor to be based on win32eventreactor
instead of on selectreactor.  Ideally, this would be done generally so that
wxreactor could be used with /any/ other reactor, rather than being specific
to win32eventreactor, but it wouldn't necessarily have to be done that way.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Question

[Jean-Paul Calderone]

On Sun, 31 May 2009 19:12:56 -0400, Brandon McGinty brandon.mcgi...@gmail.com 
wrote:
I'll give that a shot...unfortunately, the data is only accessible via
an irc server and protocol.
Hopefully I can gleen some useful information about cross-protocol
communication by lokng at the source.

http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeinputononeconnectionresultinoutputonanother
 answers a question very similar to yours.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] ESMTP extension

[Jean-Paul Calderone]

On Fri, 29 May 2009 21:08:59 -0500, Nicolas Toper nto...@gmail.com wrote:
No problem, just DIGEST-MD5. No need for PLAIN or LOGIN?


I think it handles PLAIN and LOGIN already (and CRAM-MD5).  Maybe we're
not talking about the same API?  I was thinking about the ESMTP server.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] buildProtocol never called

[Jean-Paul Calderone]

On Sat, 30 May 2009 18:36:16 -0500, travis+ml-twis...@subspacefield.org wrote:
On Sat, May 30, 2009 at 06:17:54PM -0500, travis+ml-twis...@subspacefield.org 
wrote:
 The program works fine normally, and can run in the background, but if I
 invoke a daemonize() routine that turns it into a network daemon, it
 refuses to serve incoming TCP connections.  Actually the TCP connection
 is made, but the software never responds to it.

 Specifically, my twisted.internet.protocol.Factory instance is created,
 but buildProtocol is never called.

 Before I dive deeply into debugging the software, I thought I'd ask if
 anyone knew off the top of their head any reason why Twisted might not
 act the same if it was daemonized (no controlling terminal, no
 stdin/out/err, forked as a background process).

Here's some behavior I've gathered through debugging.  Note that all of
the daemonization occurs before starting the reactor.

Creating the reactor at all may create a file descriptor which is necessary
for its operation.  Closing these will very likely cause problems.

Also, Twisted has daemonization features already which are known to work
well. ;)  Why aren't you using these?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] ESMTP extension

[Jean-Paul Calderone]

On Fri, 29 May 2009 01:00:05 -0500, Nicolas Toper nto...@gmail.com wrote:
Hi,

I am using SMTP-AUTH with Twisted. I have extended smtp.mail to handle
LOGIN and PLAIN on the server side (the version I have handles only
CRAM-MD5). I am adding currently DIGEST-MD5.

Do you want a patch?

If yes, what is the best way to submit a patch (I could not find it on
the wiki)?

Hi Nicolas,

A patch adding DIGEST-MD5 support would be great.  You can find details on
how to contribute here:

  http://twistedmatrix.com/trac/wiki/TwistedDevelopment

You might also find twisted.cred.credentials.DigestedCredentials useful.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] reactor.stop() and deferToThead

[Jean-Paul Calderone]

On Fri, 29 May 2009 14:22:43 +0200, Thomas Jakobsen 
thomas.jakob...@alexandra.dk wrote:
Hi

It seems that things being deferredToThread continue to run even
though reactor.stop() is called.

Threads cannot be interrupted.  They will run until they function they are
running returns.


Output from the example below is:

stuff1 finished; stopping reactor
stuff2 finished

Is there a way to abort the remaining execution of stuff2 in this
case? It would be handy if, say, some exception happens in in stuff1
that causes the execution of the remaining stuff2 to be meaningless.

You can make stuff2 cooperate with stuff1 so that it returns earlier.

 [snip]

def stuff1():
time.sleep(2)
print stuff1 finished; stopping reactor
reactor.stop()

Also, you're calling reactor.stop() in a non-reactor thread here.  This is
not allowed.  You must call it in the reactor thread, along with almost
every other API.  Try reactor.callFromThread(reactor.stop), instead.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted protocol as Django data source

[Jean-Paul Calderone]

On Wed, 27 May 2009 14:39:45 -0400, Dan charn...@muohio.edu wrote:
Hi all,

I have an application written in Twisted which multiplexes data from a
variety of pieces of specialized hardware, and I'd like to make the
live stream of this information viewable through a webpage.
Unfortunately, on our server, port 80 is already taken by an Apache +
mod_python + Django stack with which we host a number of webpages.

 [snip]

The options I've thought of include using Twisted's builtin web server
a nonstandard port, mod_proxy, or writing a Django app which just
polls a database repeatedly and altering my Twisted app so it
repeatedly does an UPDATE. All of these solutions feel hacky to me, so
I'd like to ask you, the list, how you'd go about doing this.

Another option is to have Django talk to a Twisted process via some RPC
mechanism that won't require you to use Twisted in the Apache process.
For example, XML-RPC.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] First time install of Twisted fails

[Jean-Paul Calderone]

On Wed, 20 May 2009 09:12:28 -0500, Uncle Roastie roas...@rochester.rr.com 
wrote:
I'm trying to do an install of Twisted for Python 2.5.4
on Windows XP.  I installed PyCrypto, OpenSSL for Windows,
and PyOpenSLL.   When I enter the Python interpreter
and say

 import OpenSLL

I get the following error:

The procedure entry point SSL_CTX_set_info_callback could not be
located in the dynamic link library ssleay32.dll.

The DLL lives inside the OpenSLL bin directory, which is also on the search
PATH.

Any ideas?


This question would be better directed at the pyOpenSSL mailing list, I
think.  pyOpenSSL is not part of Twisted, and this seems to be a pyOpenSSL
installation issue.

Additional important information you should include:

  - what version did you try to install?
  - which package did you try to install (full file name or download URL)?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] offer of adding IRCD support to words.protocol.irc

[Jean-Paul Calderone]

On Tue, 19 May 2009 18:17:01 +0100, Patrick Thomson patrick.thom...@gmail.com 
wrote:
I'm currently about to embark on building something from twisted which
has an IRCD, and I have found the capacity to run IRC servers is
lacking, in contrast to the excellent client/bot backend code.

I've found several other twisted projects already implement an IRCD,
and whilst I'm reinventing the wheel, I may as well go the extra mile
to get my code suitable for inclusion.

Hi Patrick,

Cool!

I want to point you at what happens when you run this command:

  twistd -n words --irc-port 6667 --group irctest 
--auth=memory:username:password

Ideally, an IRCD in Twisted would be based on this (which certainly needs
some work).

I wouldn't be asking for a lot, just some general guidelines (e.g. I
know my patch probably won't be accepted
without test code as well).

Tests are probably the most important single thing.  We also ask for full
docstring coverage (ie, docstrings for every class, method, function).  It
would also be best if you could structure your contributions incrementally.
Twenty patches of a couple hundred lines each is vastly preferable to one
patch of several thousand lines.

You can find much more detailed information on the wiki,

  http://twistedmatrix.com/trac/wiki/TwistedDevelopment

If anything there needs clarification, please ask. :)

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] help using deferred

[Jean-Paul Calderone]

On Mon, 11 May 2009 12:24:54 +0200, Luigi Conte luigiandcosoluti...@gmail.com 
wrote:
Hi people,
I'm luigi and I'm a student at Politecnico di Milano, Italy. I'm working for
a graduate thesis on a framework called usher. It provides an API to do
connect e command operations. It manages all those calls with a defferred
object (i think). In fact when I call the connect it returns me a defferred
object. All that i want to know is how to call the other operations with
this object and how to know when an operation is completely done.
The procedure I call is simple: connect method, start method many times,
stop method, disconnect method. I want to call these methods sequentially
because I have to fallow a list of commands so I have to be sure the
previous methods are already been invoked. I'll post the main methods hoping
someone could help me in how to add the method in the deferred and how to
know when it is completely invoked.

The Deferred tells you two things.  It tells you when an operation has
completed and it tells you the result of the operation.  It tells you
*when* by calling your callback function at that time.  It tells you
what *result* by the value it passes to that callback function.

So if you want to connect, then do the start operation many times, then
stop, then disconnect, you have some Deferreds and a simple recursion-
or loop-like structure.  For example,

# First, set up the connection.
taskDeferred = connect(...)
def cbConnected(connectResult):
# This function will run once the Deferred returned by `connect´
# fires.  Here, start something.
return start(...)
taskDeferred.addCallback(cbConnected)

def cbStarted(startResult):
# Now the Deferred returned by `start´ has fired.
if someStopCondition:
# If we decide it is time to stop, then stop.
return stop(...)
# Otherwise, run `start´ again
startAgainDeferred = start(...)
# And call this callback again with this new call's result.
startAgainDeferred.addCallback(cbStarted)
return startAgainDeferred
taskDeferred.addCallback(cbStarted)

def cbStopped(stopResult):
# The only exit from the earlier loop is through the code path
# which calls `stop´.  This callback is called when the Deferred
# returned by that call fires.  Do the disconnect now.
return disconnect(...)
taskDeferred.addCallback(cbStopped)

def cbDisconnected(disconnectResult):
print All done
taskDeferred.addCallback(cbDisconnected)

# It's possible something will go wrong in the above.  To make sure any
# exceptions are reported, add a logging errback at the very end to
# display whatever might go wrong.
taskDeferred.addErrback(twisted.python.log.err)

Hope this helps,

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] help using deferred

[Jean-Paul Calderone]

On Mon, 11 May 2009 16:55:40 +0200, Luigi Conte luigiandcosoluti...@gmail.com 
wrote:
I'm sure you're saying only good things but I'm so inexpert in twisted that
I can't understand very well what to do.
I'll try to explain all that I want to do and how I did it wrongly:
first I have to call the connect method from the api and it returns a
deferred so I do:
in conn_to_ctrl i have:
d = api.connect(...)
return d

now I do some operations with config files to call many times the start
method from the api (I have to start many virtual machines) and the
api.start returns a deferred so what have I to do now?
I do so:
in examinecfg i have:
# tha same d that i used to add api.connect!
self.d.addCallback(api.start, (method_args))
return d

Then I have to call the disconnect method from the api. But i tried to print
out something to fallow the executing but I only see printing the first
method that call the connection. after that the process ends.

In the main I have:
defer = conn_to_ctrl()
or have I to add to defer also the examinecfg?

thank you very much

I'm not sure what you're doing wrong.  It's hard to understand mixed up
snippets and fragments of code.  Please:

  - don't top post
  - share complete, runnable examples (preferably minimal, too)
  - share the failure you're seeing - possibly including a traceback, or a
description of the behavior that the program exhibits when you run it,
and an explanation of how this differs from what you want

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted HTTP 1.1 Web Client

[Jean-Paul Calderone]

On Fri, 8 May 2009 13:06:42 +0100, Reza Lotun rlo...@gmail.com wrote:
Hi,

Just curious, but what's the roadmap for the new web client
(http://twistedmatrix.com/trac/ticket/886)? As I understand it only
the finalization of a public api remains?

That, as well as the implementation of higher-level APIs on top of it.
For example, APIs which automatically handle redirects for you, or cookies,
timeouts, caching, etc.

Chris did a little work towards timeout support at the last sprint.  The
idea is that at least some of these higher-level APIs should be implemented
before we declare the low-level interface complete and public.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] How can I change max simultaneously connection in twisted

[Jean-Paul Calderone]

On Tue, 05 May 2009 20:22:22 +0800, 孙绍轩 sunshaox...@cdeledu.com wrote:
孙绍轩 写道:
 [snip]
I've found out how to solve the problem!
I've changed python from 2.4 to 2.5, the problem has gone. It's a bug in
twisted?

No, a bug in Python.

http://mail.python.org/pipermail/python-bugs-list/2006-June/033701.html

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] deferToThread help

[Jean-Paul Calderone]

On Wed, 29 Apr 2009 11:45:09 -0700, Minesh Patel min...@gmail.com wrote:
Sorry if this is a newbie question.

I have a blocking function that I defer to thread, let's say 'foo' and
I would like to add a callback that gets called after say 10 minues
after Thread finishes

def foo():
   # Blocks for a while

def bar():
   # Do some stuff

d = threads.deferToThread(foo)
d.addCallback(bar) # How can I add timing to the callback???

Currently I have the following code which works but doesn't account
for the threads execution time:

reactor.callInThread(self.bootIso, self.iloIp, pathToIsoRename)
# 10 minutes sleep
d = task.deferLater(reactor, 600, bar)


You just need to combine the relevant parts of these two solutions.  First,
the deferToThread call which will let you know when the task in the thread
has completed (this is not when the thread exits, because deferToThread uses
a thread pool and threads are re-used for many tasks):

fooComplete = deferToThread(foo)

Next, when `foo´ is finished, you want to wait 10 minutes.  You had the
right idea with deferLater, but you don't want to call it until `foo´ is
finished:

def waitTenMinutes(result):
return deferLater(reactor, 600, lambda: result)
fooComplete.addCallback(waitTenMinutes)

Finally, after a ten minute delay, you want to call `bar´:

fooComplete.addCallback(bar)

Since a Deferred has a chain of callbacks which are called one after the
other, and since return a Deferred from a callback causes things to wait
for the result of that new Deferred before moving on to the next callback
in the original chain, combining deferToThread, deferLater, and bar as
above will give you the desired behavior.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Polling from Twisted

[Jean-Paul Calderone]

On Tue, 28 Apr 2009 14:57:31 -0300, Juanjo Conti jjco...@gmail.com wrote:
I'll 400 clients in the first deploy and some thousands later. Do you
think are many?

You might want to consider a more efficient protocol.  For example, when
a client connects, you could tell it that you want it to send you its
status once every thirty seconds.  Then you don't need to send the request
to each client twice a minute, which saves you the transmission overhead
and the time-based scheduling overhead.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Upcoming Twisted Sprint in Cambridge, MA

[Jean-Paul Calderone]

On Mon, 20 Apr 2009 15:35:43 -0400, Jean-Paul Calderone exar...@divmod.com 
wrote:
Hello All,

I'd like to organize another Twisted sprint in Cambridge, MA in May.  As
with previous sprints, this is an all day event, most likely hosted at the
Divmod office in Porter Square, for Twisted hacking.  Chris Armstrong has
raised plugin system improvements as a possible topic.  Other topics are
welcome as well, of course.

It's been a couple months since our last sprint.  I think it would be nice
to have a sprint early in May and another one late in May.  How do people
feel about May 2nd or 3rd for the first sprint?


Okay, it's official.  Sprint is on for this Saturday, May 2nd.  I'll be
there by 10 AM and probably be around until at least 6 PM (likely later).

Anyone who would like directions or other information about how to
participate, please feel free to contact.

I'll also propose a couple days for the next one.  Does anyone have a
preference between May 23rd and May 30th?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Upcoming Twisted Sprint in Cambridge, MA

[Jean-Paul Calderone]

Hello All,

I'd like to organize another Twisted sprint in Cambridge, MA in May.  As
with previous sprints, this is an all day event, most likely hosted at the
Divmod office in Porter Square, for Twisted hacking.  Chris Armstrong has
raised plugin system improvements as a possible topic.  Other topics are
welcome as well, of course.

It's been a couple months since our last sprint.  I think it would be nice
to have a sprint early in May and another one late in May.  How do people
feel about May 2nd or 3rd for the first sprint?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] _Win32Waker

[Jean-Paul Calderone]

On Mon, 20 Apr 2009 18:16:39 -0500, Aron Bierbaum aronbierb...@gmail.com 
wrote:

I have looked into this a little more and have noticed that if I
specify a port number instead of 0 it will always bind to the
correct 127.0.0.1 address. I still don't know why this only occurs
on certain Windows machines. I will hopefully get more time to look
into this in the next couple of days.


Hmm.  Interesting.  Perhaps we could bind a different port, then.
I look forward to results from your further investigations.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Plugin startup and shutdown actions

[Jean-Paul Calderone]

On Sat, 11 Apr 2009 00:24:37 +0200, luper rouch luper.ro...@gmail.com wrote:

I would like to port my Application API based services to the twistd
plugin interface, to be able to retrieve command line options.

In the existing services, I reimplemented t.a.s.MultiService
startService() and stopService() to do some actions at startup and
shutdown.

How can I do this in a plugin ?



The top-level API for plugins is mostly a function that returns an IService
provider.  So if you have your own version of MultiService, you can just
return an instance of that, just like the one you were creating in your .tac
file, from a makeService function which is registered as a plugin of the
suitable type.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Fork/Spawn children to accept connections on the same port.

[Jean-Paul Calderone]

On Mon, 6 Apr 2009 15:25:39 -0700, Eric York ey...@apple.com wrote:
I am trying to get the highest level of performance using all of the 
processors cores on a server.


In the past, a unix app would bind/listen to a socket and then fork or 
spawn children to accept connections on that socket. I can’t see how  to do 
that in Twisted. Can someone point me in the right direction?


The solution of using a single process to accept connections and then  farm 
out work to child processes, while a workable solution, isn’t at  the same 
level of performance as children processes that are doing  their own 
accepts.


There are many ways to do this.  The common idea is to just run multiple
processes, not by allocating resources and then forking to share them, but
just by running multiple processes.  You can make this transparent to the
outside in various ways - have a load balancing port forwarded in front of
all the processes, for example.  On Linux, you can even use iptables to do
this.  It's also possible to bind a port in one process and then send it
over a UNIX socket to another process; this comes closer to the bind/fork
approach, but accomplishes the resource sharing explicitly via fd passing
rather than through fork.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] trac server error

[Jean-Paul Calderone]

On Tue, 07 Apr 2009 16:13:53 +0200, Johann Borck johann.bo...@densedata.com 
wrote:

there seems to be some serious problem with http://twistedmatrix.com/trac/
this is what the browser shows me on about one third of the requests:


Sorry.  This just happens sometimes.  It's due to some bug in trac.  I think
that it generally should only happen near the beginning of a session, but
I'm not certain.  If you have cookies disabled, this might cause you to see
it more frequently.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Stackless Twisted shocking performance

[Jean-Paul Calderone]

On Fri, 20 Mar 2009 18:16:47 +, Simon Pickles sipick...@googlemail.com 
wrote:

[snip]

As more updates are sent zone-hub, per second, the incoming 
client-hub-zone messages take SECONDS to get thru! This is with approx 50 
messages per second between a single zone and the hub.


message is a very non-specific unit of message.  For example, I just
threw together a PB demo that does 1754 messages per second.  That's a
lot more than your 50/sec, but who knows why?  You'll need to narrow down
the bottleneck a bit more precisely before there's much chance of getting
rid of it.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Stackless Twisted shocking performance

[Jean-Paul Calderone]

On Sun, 22 Mar 2009 18:32:30 -0500, Jean-Paul Calderone exar...@divmod.com 
wrote:
On Fri, 20 Mar 2009 18:16:47 +, Simon Pickles sipick...@googlemail.com 
wrote:

[snip]

As more updates are sent zone-hub, per second, the incoming 
client-hub-zone messages take SECONDS to get thru! This is with approx 50 
messages per second between a single zone and the hub.


message is a very non-specific unit of message.  For example, I just


Ahem, excuse me.  Unit of *measure*. :)


threw together a PB demo that does 1754 messages per second.  That's a
lot more than your 50/sec, but who knows why?  You'll need to narrow down
the bottleneck a bit more precisely before there's much chance of getting
rid of it.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] So how does everyone use an ORM in twisted?

[Jean-Paul Calderone]

On Tue, 17 Mar 2009 11:05:59 -0400, Ryan Lepidi ryeg...@gmail.com wrote:

I have searched a bit, and it seems as though you can't use SQL Alchemy with
twisted due to SA being thread unsafe. The only solution I found was sAsync,
but this is old and unmaintained. What is everyone else doing for database
access for a multiuser server? Is everyone really using raw sql queries or
is there some easy solution that I'm missing?

I'm not even stuck on SQL Alchemy; if there is some other ORM that works in
this case, I'd be willing to learn it.



I use Axiom (for Mantissa, a somewhat general-purpose application server,
for Quotient, a mail server based on it, for Imaginary, a text adventure
server based on it, for Blendix, a person aggregator based on it, for
Merit, a test results database, etc) most of the time.  I know some other
people are using Storm.

Overall, I think that the way all existing ORMs work (including Axiom and
Storm) is hostile towards the typical Twisted program structure (or vice
versa, or both).  So it's tough.  The approach Axiom takes is to warn you
that any database operation you perform must complete quickly or your app
will suffer, and to restrict you to SQLite, so network issues can't hurt
you.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Noob Question

[Jean-Paul Calderone]

On Sun, 15 Mar 2009 08:55:25 -0500, Shelby Ramsey sicfsl...@gmail.com wrote:

Peter,

Thanks for the assistance.  I think you and David have me on the right
path.  Just to clarify the protocol looks like this:


Hi Shelby,

You may want to consider re-using an existing protocol rather than inventing
and implementing a new one.  Protocol design has many tricky corners, and
with the wide array of existing protocols from which to select, there's is
often no really good reason to develop a new one.

Consider HTTP, PB, AMP, or XMPP, all of which have general, re-usable
implementations in Twisted.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] _Win32Waker

[Jean-Paul Calderone]

On Mon, 9 Mar 2009 09:17:14 -0500, Aron Bierbaum aronbierb...@gmail.com wrote:

I have been unable to reproduce this problem on multiple machines that
I have tested on. Also I have tried changing various network settings
on my machine without any change. Do you have any ideas what I should
be looking for?


Not really, sorry.  I know that behavior often differs subtly or grossly
from Windows machine to Windows machine, but not why.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] _Win32Waker

[Jean-Paul Calderone]

On Thu, 5 Mar 2009 16:49:15 -0600, Aron Bierbaum aronbierb...@gmail.com wrote:

I have been using a custom Qt4 reactor that derives from
PosixReactorBase. As a result it creates a _Win32Waker to allow
threads and signals to wake up the IO thread. It seems though that the
current implementation only works about half of the time. The other
half it exists with :

 File ...\Lib\python2.6\site-packages\twisted\internet\posixbase.py,
line 170, in __init__
   ReactorBase.__init__(self)
 File ...\Lib\python2.6\site-packages\twisted\internet\base.py,
line 424, in __init__
   self._initThreads()
 File ...\Lib\python2.6\site-packages\twisted\internet\base.py,
line 813, in _initThreads
   self.installWaker()
 File ...\Lib\python2.6\site-packages\twisted\internet\posixbase.py,
line 206, in installWaker
   self.waker = _Waker(self)
 File ...\Lib\python2.6\site-packages\twisted\internet\posixbase.py,
line 77, in __init__
   client.connect(server.getsockname())
 File string, line 1, in connect
socket.error: [Errno 10049] The requested address is not valid in its context

I have attached a simple test that shows that the following code does
not always return 127.0.0.1, but sometimes returns 0.0.0.0 as the
IP address.

   # Following select_trigger (from asyncore)'s example;
   server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
   client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
   client.setsockopt(socket.IPPROTO_TCP, 1, 1)
   server.bind(('127.0.0.1', 0))
   server.listen(1)
   client.connect(server.getsockname())

My current workaround just calls the following instead:

   client.connect(('127.0.0.1', server.getsockname()[1]))

Any ideas on what is really causing the error? If there is not a
better solution can this be added to trunk for future releases?


It's definitely true that you can't connect to 0.0.0.0 on Windows, and
various parts of Twisted try to deal with this in some way already.  It
isn't clear to me why that getsockname() isn't returning 127.0.0.1 though.
I expect it's due to some configuration change or third-party networking
software on the Windows machine.  Do you think you can track that down?
That will make it much easier to think about the problem and the solution.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Re:how to pass on the connection failed or connection lost error

[Jean-Paul Calderone]

On Thu, 5 Mar 2009 15:49:31 -0800 (PST), khawar hasham 
hasham_james_...@yahoo.com wrote:

Hi,

I tried to use this suggestion but I could not make it work.

here is the test I am using

in my application class
def callSend(self, msg):
  plugin.send(msg)

Now in plugin class
def send(self, msg):
  print 'before call'
  threads.blockingCallFromThread(reactor, remotesend, msg)
  print 'after call'

def handleError(self, error):
  error.raiseException()

def remotesend(self, msg):
 deferred = defer.Deferred()
 reactor.connectTCP('localhost',,myfactory)
 deffered.addErrback(self.handleError)
 return deffered

now myfactory class .. clientConnectionFailed
def clientConnectionFailed(self, connector, reason):
  reason.raiseException()


now the problem is, code has become synchronous as code is waiting after 
threads.blockingCallFromThread, how to fire the defer returned from 
'remotesend'. Do i need to fire it or twisted will take care of it.


You probably need to fire it.  You created it, so it's your responsibility.
I don't know when you want it to fire, though.  What is it supposed to
represent?


I think I can't refer this deferred object since it is being passed to 
threads.blockingCallFromThread function on which I have no control.


Not only that, but you can't do anything with it outside the reactor thread.



any suggestions



What do you want to happen?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] sending large files from web2 http server

[Jean-Paul Calderone]

On Thu, 05 Mar 2009 16:20:27 +0100, Markus Wanner mar...@bluegap.ch wrote:

Hi,

I'm trying to stream longish data via web2, but experience sudden stalls
in data transfer, followed by a connection abort after a certain
timeout. I can't completely reproduce the issue, yet, but figured that
the size of the blocks I'm returning via the stream's read method
affects the failure rate (an stream.IByteStream implementation).

I've tried with chunk sizes between 1 MiB down to 256 bytes. At that
rate, I suddenly get the following unhandled errors, which don't seem to
have to do much with my code.


I haven't looked at how web2 handles IByteStream providers, but my first
guess would be that this is an example of a somewhat common bug where
Deferreds are chained to an arbitrary length based on application data
and when there's too much application data, the chain gets too long (a
limit imposed by how much recursion is possible when unwinding the chain),
and you get this failure.

Alternatively, the bug I describe above might somehow have gotten into
your application code, rather than into Web2.  As you say, the stack
trace doesn't point to your code, but it doesn't point to Web2 code
either, and it's common in cases like this for the traceback to just
point at the implementation of Deferred, since that's where the bug
(or lack of feature, whatever) *really* is.

The main focus of web development in Twisted now is Twisted Web, not
Twisted Web2.  However, if you can supply a patch which fixes this
(along with unit tests, please :), then we'd be happy to apply it.

James Knight or David Reid might have a suggestion about where in the
code to start looking for the problem.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted developers: please donate your time!

[Jean-Paul Calderone]

On Mon, 02 Mar 2009 09:08:00 -0500, Itamar Shtull-Trauring 
ita...@itamarst.org wrote:

On Tue, 2009-02-24 at 21:25 +1300, Michael Hudson wrote:

2009/2/24 Itamar Shtull-Trauring ita...@itamarst.org:
 As part of the TSF's fundraising efforts, we are trying to get upfront
 donations of time you will spend developing Twisted. It will then get
 matched by a donor, if we are successful in getting this grant. So if
 you're planning on working on Twisted anyway this year, your work will
 count twice as much!

 So, if you're interested, please reply, saying something like I will
 spend two weeks working on Gopher support over the next year.

I don't entirely understand, but I will commit to spending the
equivalent of two working weeks as a general review monkey over the
next year.  Is that the sort of thing you wanted to hear?


Yes - thanks to you and all thee rest of the people who are
volunteering! Anyone else willing to step up?



I will spend the equivalent of four weeks over the next year working on
ticket triage, code reviews, Lore, and Web.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] newbie problem with SMTPClient

[Jean-Paul Calderone]

On Mon, 2 Mar 2009 15:57:17 -0500, Dave Britton d...@davebritton.com wrote:

I love the idea of twisted but I think I must have a twisted learning 
disability, as I have gotten nowhere in what ought to be a simple matter.

I need to send out emails to small groups from my apache server running a 
python cgi using mod_python, but my hosting service doesn't have a MTA. Instead 
of learning how to install and configure exim I thought I would use twisted to 
make a simple mail client. I started with the tutorial example that appears at:


It may be that installing exim is actually a better idea.  In order to get
reliable message deliver, you'll need to handle transient failures.  That
means persisting state over time (as long as several days) and performing
redelivery attempts.  However, if you don't mind losing outgoing messages
when there is a transient failure...


[snip]

This nicely looks up the right MX record and sends out an email, just what I 
need. Now I want to expand it to allow me to give it a list of email addresses 
to send the message to (not just call this same routine multple times, which 
seems wasteful and slow and doesn't use twisted's power to process the multiple 
emails in multiple threads), but I'm having terrible trouble figuring out how 
to do that, which tells me I'm missing a paradigm somewhere, there's something 
I'm not getting.


There is a reason that just calling the top-level function you've written
once for each email might be wasteful, but I don't think it's the reason
you're thinking.

If you just naively call getMailExchange(host).addCallback(cbMailExchange)
once for each email, then you will get parallel processing.  Almost as soon
as you call most Twisted APIs, they'll start an operation (to be precise,
many of them start it immediately - before they even return - and others
start it when you allow program execution flow to return to the reactor).
So if you call getMailExchange in a loop, each iteration of the loop will
start a new operation and they'll all run in parallel.  That seems like
just what you're after.

The reason this isn't the most efficient solution is that your list of
email addresses might contain two addresses which have the same mail
exchange host.  In this case, you could connect to that mail exchange
once and address your message to both addresses and then deliver the
message body to the mail exchange just once.  This is why getMailTo
returns a list.

So I think your other questions aren't directly relevant to this problem,
but I'll answer them anyway, since they're good questions.



Trouble 1 is figuring out the right way to pass additional parameters to 
callbacks. Is this right:

Dosomething(that-returns-a-deferred).addCallback(Then-do-the-next-thing, 
extra-parameter1, extraparameter2)

The function Then-do-the-next-thing() will receive the deferred returned 
results from Dosomething() as its first argument, and extra-parameter1 and 
extraparameter2 as the next two. That is as if calling:
Then-do-the-next-thing(Result-returned-by-Dosomething(),extra-parameter1, 
extraparameter2). Have I got this correct?


Yes, this is right.


So, if this is right, then where do I want to put the additional argument that 
contains the next email address to send, if I iterate through the list and hand 
each one to the email sending process like this:

elist=['ad...@domain.com','ad...@nextdomain.com'...]
for e in elist:
 e2={'mxhost':'','toaddr':e}
 getMailExchange(e2).addCallback(cbMailExchange)



I'm not sure why you switched to a dictionary here.  Ignoring that, the
biggest potential problem with this code snippet is that it creates a
Deferred (the one returned by getMailExchange) and then drops it on the
floor (albeit after adding a callback).  You'll almost certainly want to
build up a list of Deferreds in cases like this, and then use something
like twisted.internet.defer.gatherResults to find out when they've all
fired.  Otherwise, you don't really know when your list of operations has
completed.

As to where to put extra arguments for tracking which email addresses to
send the message to next, I don't see why you'd want that in this case.
Your for loop iterates over all the addresses, so none of your callbacks
should need to know about any more addresses.  That is to say, by the end
of the for loop, there are no more addresses to which the message needs to
be sent - you've started sending to all of them already.


In the tutorial, getMailExchange() is passed just the domain of the addressee, 
and the sending out of the email happens when the callback returns the MX 
exchange. I changed that to split the email address, so now it returns both the 
full address and the mx:

def getMailExchange(addr):
 host=addr.split('@')[1]
 def cbMX(mxRecord):
  return [addr,str(mxRecord.name)]
 return relaymanager.MXCalculator().getMX(host).addCallback(cbMX)


At this point I can't figure out how to get the email address passed to 
wherever 

Re: [Twisted-Python] Bug in twisted.names.client

[Jean-Paul Calderone]

On Mon, 2 Mar 2009 23:33:57 +0100, Nicolas Toper nto...@gmail.com wrote:

Good point, you are right. Thanks :)

I can still commit a patch: really there just need a finally clause at the
end of maybeParseConfig to close the file AFAI


A patch (with unit test :) would be quite welcome.  Please open a ticket
at http://twistedmatrix.com/trac/ (you'll need to register an account
first) and attach it there.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Development Process Failure

[Jean-Paul Calderone]

Hey all,

Recently I've been bothered by a systematic shortcoming of the Twisted
development process.  When changes are made in response to a review,
they are generally made in a way which is difficult to inspect.  The
value of a review is lost if the valid points raised by it are not
addressed before the changes are applied to trunk.  Often, there's no
problem here, but it sometimes happens that /not/ all of the review
points are addressed before a ticket is re-submitted for review.  In
these cases, one of two things happens.  The re-reviewer might go back
to the previous review and verify that all of the points raised in it
have been addressed.  This is a time consuming process, though it can
be made easier if the author responds to each review point (and even
easier if a changeset is linked in each response).  Or, the re-reviewer
might assume that the author addressed all points and just look over the
branch for anything missed in the previous review.

In either of these cases, time is being wasted, but in the latter case,
time is being wasted /and/ bad code is being added to trunk.  I would
like to consider how we might address this shortcoming of the development
process.

One possibility is to explicitly adjust the review guidelines and direct
reviewers always to verify that previous review points have actually been
addressed.  What ideas do other people have?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted slow in Mac OS X fast on Windows

[Jean-Paul Calderone]

On Fri, 20 Feb 2009 11:11:24 -0800, Brian Granger ellisonbg@gmail.com 
wrote:

Hi,

We use Twisted extensively in the IPython project for parallel and
distributed computing.  We have an extensive test suite that uses
trial to test our servers and clients.  Everything works
great...but...the test suite takes much longer to run on OS X than
windows.  Here is what I mean:

1.  2.53 Ghz Intel Core 2 Duo, Mac OS X 10.5.  Test suite takes 67 seconds

2.  *Same machine*, running Windows XP as VM in VMWare.  Test suite
takes 17 seconds.

I have verified that all the tests are being run on Windows.  This is
very surprising to me as I would expect, if anything, the Windows VM
to be a bit slower, not 3x faster.  This is the same hardware, the
same code, the only thing that I can think of that is different is the
reactor implementation.  I am using the default reactor on both
platforms.

Questions:

* Is the Windows reactor really that fast?


Probably not.  But it depends what your test suite does.


* Is the default (I think it is select) reactor on OS X really that slow?


Probably not.  But it depends what your test suite does.


* Why else would it be that slow on OS X?
* What can I do to speed this up?


The only thing that comes to mind is that some hostname lookup functions
used to be pretty slow on OS X.  I have no idea if this is still the case.
I suggest profiling the suite on OS X and Windows and comparing the results
to get a better idea of where to look for the problem.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Debugging Twisted

[Jean-Paul Calderone]

On Fri, 20 Feb 2009 20:15:02 +0100, Luper Rouch luper.ro...@gmail.com wrote:

Jean-Paul Calderone wrote :
On Fri, 20 Feb 2009 19:56:54 +0100, Luper Rouch luper.ro...@gmail.com 
wrote:
I would like to investigate on the conch error I mentionned in my previous 
message (KeyError when an errback is triggered within a conch TCP 
tunnel).


My problem is Twisted catches the exceptions and the debugger won't stop 
on them, how could I avoid that ? Is there a debug reactor ?


Try the --debug option to twistd.

Jean-Paul


My application is a client, how can I run it with twistd ?


See http://twistedmatrix.com/projects/core/documentation/howto/tap.html

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Sprint in February

[Jean-Paul Calderone]

On Thu, 5 Feb 2009 16:56:51 -0500, Jean-Paul Calderone exar...@divmod.com 
wrote:
On Mon, 26 Jan 2009 18:10:32 -0500, Jean-Paul Calderone exar...@divmod.com 
wrote:

Hello All,

I'd like to organize another Twisted sprint in Cambridge, MA in February.
As with previous sprints, this is an all day event, most likely hosted at
the Divmod office in Porter Square, for Twisted hacking.  The two topics
which have been raised thus far for this sprint are documentation - in
particular, looking at Twisted documentation from a high-level and coming
up with a coherent overall strategy for improving it - and deprecation
(looking for old, bad APIs in Twisted which we should get rid of).  Of
course, other topics are welcome as well.


The sprint will be held on February 22nd.  All are welcome.  If you need
details on how to get to the sprint, please contact me off-list.



Just a reminder, the sprint is this Sunday.  If you don't come, Twisted
will be sad.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] mind, twisted.cred and HTTPAuthSessionWrapper

[Jean-Paul Calderone]

On Thu, 19 Feb 2009 19:39:48 +0100, Esteve Fernandez est...@sindominio.net 
wrote:

On Thursday 19 February 2009 18:57:29 Jean-Paul Calderone wrote:

On Thu, 19 Feb 2009 18:47:37 +0100, Esteve Fernandez est...@sindominio.net

wrote:

On Thursday 19 February 2009 18:28:59 Jean-Paul Calderone wrote:
 What kind of extra information are you trying to pass?  Generally, all
 information belongs with the avatar object.  The mind should be used to
 interact with the protocol's notion of the user (as the avatar is used
 to interact with the realm's notion of the user).

Our checker generates a session per successful login, and all subsequent
 calls use that session to authenticate requests. The way I thought for
 using a mind is to create an empty object and populate it with the remote
 session object once the user has logged in successfully.

Is a mind the right place to put that session?

It doesn't seem like a good use of the mind to me.  I'd put the session
inside the avatar.  You may need a wider interface than IResource for
this, if you want to inspect the session from anywhere other than the
avatar itself (since the avatar's interface is IResource in this case),
or you may just want to use the session in order to implement getChild
and render appropriately.


But, in our case, requestAvatar can only return something based on what the
checker returns. I mean, requestAvatar receives an avatarId (as a result of
Checker#requestAvatarId), a mind and a bunch of interfaces, but the realm
doesn't have any other information.

What our checker does to authenticate an user, is to call a method on a remote
service and get back an object (a token) that represents the state of a
conversation between a client and that remote service. Then, the client may
issue more remote calls using the previously received token.

We're using Twisted.web only as a front end to a remote service, and we'd like
to make it as decoupled as possible, as there will be other frontends.


What if you use the token as the avatarId?



I tried to mimic PB's behaviour, in the sense that (as the cred documentation
puts out):

it is an object which serves an important role - any notifications which are
to be relayed to an authenticated client are passed through a 'mind'. In
addition, it allows passing more information to the realm during login in
addition to the avatar ID.


This is technically true but as far as I know no one has ever actually used
it that way.  I'm trying to think of an authentication scheme which would
benefit from it (let alone require it), but I'm drawing a blank.  PB just
has enough expressive power already to really need this.



Actually, our resources are pretty dumb, as they only issue remote calls to an
external service and don't hold any information. And, since our realm can't
figure out whether a user can issue a particular call or not without actually
sending a token to the remote service, a mind object seemed the right place
to me... but I'd happy to be wrong :-)

Anyway, sorry if I sound as if I'm trying to get a positive opinion for a
decision we already made :-P it's just that I don't see how the other options
would completely fit our architecture (which could be broken in the first
case, of course).


No problem.  Hope this has helped in some way, at least.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] mind, twisted.cred and HTTPAuthSessionWrapper

[Jean-Paul Calderone]

On Thu, 19 Feb 2009 20:22:30 +0100, Esteve Fernandez est...@sindominio.net 
wrote:

On Thursday 19 February 2009 19:46:59 Jean-Paul Calderone wrote:

But, in our case, requestAvatar can only return something based on what
 the checker returns. I mean, requestAvatar receives an avatarId (as a
 result of Checker#requestAvatarId), a mind and a bunch of interfaces, but
 the realm doesn't have any other information.

What our checker does to authenticate an user, is to call a method on a
 remote service and get back an object (a token) that represents the state
 of a conversation between a client and that remote service. Then, the
 client may issue more remote calls using the previously received token.

We're using Twisted.web only as a front end to a remote service, and we'd
 like to make it as decoupled as possible, as there will be other
 frontends.

What if you use the token as the avatarId?


But the general contract for requestAvatarId is to return a string identifying
the user. From ICredentialsChecker#requestAvatarId documentation:

Returns - a Deferred which will fire a string which identifies an avatar, an
empty tuple to specify an authenticated anonymous user (provided as
checkers.ANONYMOUS) or fire a Failure(UnauthorizedLogin). Alternatively,
return the result itself.

If so, our problem is that the token is an object that contains more
information (a timestamp, a cryptographic signature, etc.), so serializing
and deserializing it everytime we need that information is a bit cumbersome.
The remote service checks that token everytime it receives a call, and
responds with an appropiate response.


The convention is for avatarId to be a string, but as the anonymous case
indicates, that's not strictly required. :)  The real requirement is that
the realm be able to interpret the avatarId.  The way this generally works
is by making it a username string, but you're in charge of the realm, so it
can be whatever you like.  Of course, this makes it harder to re-use either
your checker or your realm with different components, but sometimes that's
okay.  It might be nice if this were explained in some document somewhere
but I don't think the current cred document should cover it, since going
into complex and subtle points like this tends to make it harder to get
the basic idea.  An advanced cred howto of some sort might be merited.


I tried to mimic PB's behaviour, in the sense that (as the cred
 documentation puts out):

it is an object which serves an important role - any notifications which
 are to be relayed to an authenticated client are passed through a 'mind'.
 In addition, it allows passing more information to the realm during login
 in addition to the avatar ID.

This is technically true but as far as I know no one has ever actually used
it that way.  I'm trying to think of an authentication scheme which would
benefit from it (let alone require it), but I'm drawing a blank.  PB just
has enough expressive power already to really need this.


In that case, (although we might not actually need a mind in the end), would
PB have the same problem with HTTPAuthSessionWrapper? The mind argument is
always None (in HTTPAuthSessionWrapper#_login):


I'm not quite sure what you mean here - how should I equate PB and
HTTPAuthSessionWrapper?


   def _login(self, credentials):
   d = self._portal.login(credentials, None, IResource)
   d.addCallbacks(self._loginSucceeded, self._loginFailed)
   return d

So, I think HTTPAuthSessionWrapper should have a method (e.g.
createMind(self)), which would return None by default and that could be
overridden by subclasses if they need to return something else. I'll gladly
file an issue, provide a patch and a test case, if you think it's a valid
issue.


It probably *is* a good idea to allow the mind to be created somehow.  It
was only omitted because no obvious use-cases presented themselves at the
time.  I'm still not really convinced that this use case is a good one
though, so I'm not sure I'd be in favor of adding the hook now in order
to support this. ;)

In general, the mind feature of cred is underutilized, underdocumented,
and poorly understood.  Fixing these problems is great, but it may also
be the case that the mind is such an obscure use-case that most people
really just don't care about it.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python