Re: [Twisted-Python] Developer docs should be updated on wiki when steps changed in code?

[Glyph]

> On Mar 1, 2021, at 5:30 AM, Adi Roiban  wrote:
> 
> 
> 
> On Wed, 3 Feb 2021 at 18:09, Glyph  > wrote:
> 
> [snip]
> 
> I can't quickly find the place where we agreed to this, but I think several 
> years ago at this point we had a discussion about moving all these docs into 
> the source tree.  (If they're on the wiki, there's no review process or even 
> a place where updates can be staged for commentary before going live and 
> becoming "official".)
> 
>> Until this is fully completed, what is the correct thing to do when there is 
>> a mismatch between docs on the wiki, and docs in the tree?
> 
> Docs in the tree always win.
> 
>> I still refer to a lot of docs on the wiki, especially for process related 
>> things,
>> so I think it would be nice if the wiki docs were kept up to date, until the 
>> day that
>> they are fully deleted.
> 
> Let's start deleting them now, and replacing them with links to the in-tree 
> docs, rather than updating them.  They've been skewing out of date for a long 
> time.  When I was looking for information about how to do a revert, I found 
> wiki docs about linking to revisions in Subversion which didn't mention 
> Github, which gives a flavor for how outdated some of this stuff is.
> 
> Thomas, would you mind doing the honors for this document?  Links for process 
> docs should probably be to https://docs.twistedmatrix.com/en/latest/ 
>  since, for process information 
> (unlike API information), trunk should be authoritative, not the latest 
> release.
> 
> 
> I don't know what wiki page we are talking about here. Any link would help.

Upthread (in the [snip]'d part), Craig referenced 
https://twistedmatrix.com/trac/wiki/TwistedDevelopment 
, and I thought we'd 
migrated some stuff from there.  If not, never mind.

> I see this wiki page and it already has a redirection: 
> https://twistedmatrix.com/trac/wiki/ReleaseProcess 
> 
> 
> My suggestion is that next time a dev needs to update a wiki page, 
> she/he/they should consider moving that page to twisted/twisted repo
> narrative docs and create a PR for that change.

Sounds good to me.

> From this wiki page https://twistedmatrix.com/trac/wiki/TwistedDevelopment 
> 
> 
> I see that Security and Review Process are not yet migrated to narrativedocs.

Does someone want to go file tickets for those?

> And I think that "Contributor Advancement Path" can be removed as we now have 
> an informal processes for giving write access to the repo.

Perhaps this proposal has been rejected but I think we should leave a stub in 
there saying that we could really use some help with a formal process.  The 
informal process is sort of necessary at the moment, but implicit processes 
tend to overlook quieter people, and people who are less sure of themselves in 
the open source social context.  However, improving this process is going to be 
some work, and we'll have to find someone willing to do it.

-g

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Klein?

[Glyph]

> On Mar 1, 2021, at 12:51 PM, Robert DiFalco  wrote:
> 
> Is this the right place to ask klein questions?

Absolutely, it's a Twisted org project.

> I'm writing a metrics plugin for Klein and I can't figure out how to inject a 
> metrics handler so that I can get route, path, duration, and status code. 
> What I'm doing now sucks because Klein and twisted interact in complex ways 
> on Failure and status codes. 
> # Replace the klein _call with the metrics generating call
> _app._call = _callWithMetrics
> Rather than replace _call with my version of _call I was hoping there was a 
> cleaner way to get the start and stop with the result code of a route 
> invocation. Thoughts?


It sounds like you should contribute a patch that makes this an 
explicitly-supported pluggable entrypoint, rather than relying on a hack.  No 
need to figure out a way to make it work with existing versions, the magic of 
open source is that you can change it :).

Feel free to ping here when it's done to remind folks to do a review.

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [Twisted-web] Twisted 21.2.0 Release Announcement

[Glyph]

> On Feb 28, 2021, at 2:27 AM, Craig Rodrigues  wrote:
> 
> On behalf of Twisted Matrix Laboratories, I am honored to announce the
> release of Twisted 21.2.0!

Thank you so much for shepherding this release to completion, Craig.  It's so 
good to have a recent release out in the world again!  Looking forward to the 
many process improvements that have landed as well, and having more frequent 
releases produced by automation!

-g



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release blocker: Use latest pydoctor release ?

[Craig Rodrigues]

On Wed, Mar 3, 2021 at 7:18 AM Maarten ter Huurne 
wrote:

>
> Yes, as far as I know the intention is to reduce the amount of
> infrastructure that has to be maintained by Twisted developers.
>
> Adi is doing the actual work for the migration; I only contribute
> indirectly by reviewing PRs that make pydoctor integrate better with
> Sphinx.
>
> There are efforts to make the output of pydoctor more user friendly.
> This is mainly done by Tristan, but I occasionally work on it as well.
> In the next major release we should have a clearer presentation of
> parameter types, more navigation links and, if it's ready in time, a
> side bar containing a page outline.
>

Maarten,

Thanks for the detailed explanation of what you, Adi, and Tristan
are working towards.  That is a lot of hard work!

I think that using the type annotations in the code for documentation,
rather than the Epydoc @type markup tag is WAY better than what we have now.

>From what you have described, there are only a few more tasks to go before
this goal is fully completed, and the docs are published with the new
pydoctor.

I'm looking forward to the end result!

--
Craig
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Tobias Oberstein]

but we have had enough difficulty keeping our CI configuration current 
based on what cloud provider is falling over this month ;-).


Yes, CI seems to be universally one of those things that is conceptually 
simple but somehow takes hours and hours to maintain.


Absolutely agreed, glad that it's not only me feeling like that=)

If I'd sum up the time I just recently spent on pip, tox, docker, github 
actions, terraform, .. the list goes on, and compare to time actually 
spent on coding, it's depressing.


for our cloud stuff with Crossbar.io, from code-to-live looks like:

GitHub branch (code change) ->
  GitHub actions (CI main) ->
GitHub master (code merge) ->
   Python wheel (CD "deploy") ->
 Docker image (CD "docker") ->

downstream repo:

   AMI (via Packer) ->
 Terraform (live)


for devices, it looks like:

GitHub branch (code change) ->
  GitHub actions (CI main) ->
GitHub master (code merge) ->
   Python wheel (CD "deploy") ->
 snap (CD "snap") ->

downstream repo:

   Yocto ->
 S3 -> CDN -> RAUC (live)



obviously, a lot of things can go wrong.

one decision we made: all CI/CD pipelines run through a common set of 
Python wheels we build.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Tobias Oberstein]

on their end.  If initially, say, crossbar and matrix would like to work 
with us to set up some kind of repeatable pattern we can suggest to 
others, that would be great.


ok, I'll take that to the team:

https://github.com/crossbario/crossbar/issues/1867

tldr: we could run our CI in 2 sets, one using pinned deps, and one with 
open-ended deps, our latest own master branches _and_ latest Twisted 
(trunk).


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Richard van der Hoff]

On 03/03/2021 18:47, Glyph wrote:


I'll take this to the Synapse team to discuss further, but we could 
probably easily arrange for one of our CI runs to install Twisted 
trunk from git instead of pypi, which might be a good start.


This is specifically the approach I'd really rather /not/ take :) and 
here's why:


 1. You want to provide stability for you contributors so that if a
problem is introduced, you don't halt development on that
unrelated feature to diagnose the upstream issue.

 2. You want to ensure that when /users/ install your software, it
works with everything that's currently released.

I'm not sure I follow this. We'd still have CI runs that test against 
*release* Twisted; I'm just proposing that we would *also* test against 
development Twisted.


Your point about not stopping development when there's a problem is well 
noted though. A CI pipeline that runs on a timer might work fine. I'll 
discuss with the team.



but we have had enough difficulty keeping our CI configuration current 
based on what cloud provider is falling over this month ;-).


Yes, CI seems to be universally one of those things that is conceptually 
simple but somehow takes hours and hours to maintain.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Glyph]

> On Mar 3, 2021, at 4:58 AM, Jean-Paul Calderone  
> wrote:
> 
> Broadly, I agree.  But not with this part.  It seems like there is clearly a 
> trade-off that is better for everyone.  The trade-off represented by #1298:
> Breaks application code without providing any new functionality or fixing any 
> bugs
> Captures a long list of TODOs as an arbitrarily complex collection of sources 
> spread across the entire codebase
> All the work of addressing the problems still remains to be done 
> Contrast this with any basic ratchet-style approach (for example, capture the 
> list of mypy errors, then allow any PR that either removes some or doesn't 
> add any new ones):

In principle, I agree.  And some coarse-grained ratchets are absolutely worth 
doing, when they're supported by the tools - for example, we have one of these 
with the way we're handling `disallow_untyped_defs` right now.  I wish tools 
like Mypy would build in ratcheting infrastructure that was easy to use and 
convenient to deploy.

But in practice this type of ratchet involves a "# type: ignore" at every 
single call site, as well as delaying shipping stubs or maintaining separate 
stubs.  As well as building the maladaptive habit that #type:ignore or 
cast()ing your way out of type errors is the right way to address them in 
individual features and getting every code reviewer used to that.
> At the outset, no application code breaks because nothing has actually changed
> As work towards fixing the TODOs progresses, if it does break application 
> code then at least applications have a chance at some benefit
> As the work towards fixing the TODOs progresses, maintainers can easily look 
> up what issues remain by consulting the list of errors that feeds the 
> ratchet-style job.
> The exact mechanism for the ratchet isn't the point here.  Maybe Mypy plays 
> more nicely with inline comments telling it to ignore something, I don't 
> know.  The point is:

The "exact mechanism for the ratchet" introduces enough hard problems that it 
inevitably delays tooling adoption for months or years, reduces the benefits of 
the tooling adoption (what Twisted users really want are comprehensive stubs, 
less than Twisted to type-check internally, I think, and this lets us ship them 
sooner in at least a mostly-usable state).
> A centralized list of stuff left to do is better than a mess smeared across 
> the whole codebase
> If we're going to risk breaking applications we should at least try to offer 
> some value to them
> We shouldn't make applications deal with every change twice when we could 
> just disturb them once

Overall I very much appreciate your criticism and I wish that I could provide 
clear and concrete steps towards a functioning ratchet for all of these types 
of tools, but in practice the minor issues caused by the boil-the-ocean 
approach are an order of magnitude easier to address (and cheaper to leave 
un-addressed) than the effort required to build ratchets for our contributors 
that aren't a total nightmare to maintain and interact with.

Case in point: running `black` was a giant earthquake in the codebase but the 
pain associated with that was a drop in the bucket compared to the lingering, 
years-long nightmare of our attempt to build a style ratchet with 
twistedchecker.  In principle what we were trying to do with twistedchecker was 
very simple, but the practicalities ate us alive.

You could of course quite rightly argue that there were multiple conflated 
issues at play there!  But that's also the point: there's always a lot of 
complexity that comes along with trying to do codemod migrations "the right 
way" when that's going against the grain of the rest of the ecosystem.

I don't love that we've foisted this off onto our users, but I suspect Tobias 
spent a lot less time fixing the bug here than we've spent writing emails about 
it at this point ;-).

I look forward to the day where I'm wrong about all of this and every new code 
quality tool comes with a super easy-to-deploy ratchet, though.

-g

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Glyph]

> On Mar 3, 2021, at 8:25 AM, Richard van der Hoff  wrote:
> 
> 
> On 03/03/2021 08:07, Glyph wrote:
>> 
>> If dependencies could start testing against Twisted trunk in some capacity, 
>> we could get notified close to when unintentionally breaking changes occur, 
>> and dependencies can let us know well before the release happens, and we can 
>> either revert or they can fix things if the error is on their end.  If 
>> initially, say, crossbar and matrix would like to work with us to set up 
>> some kind of repeatable pattern we can suggest to others, that would be 
>> great.
> 
> I'll take this to the Synapse team to discuss further, but we could probably 
> easily arrange for one of our CI runs to install Twisted trunk from git 
> instead of pypi, which might be a good start.

This is specifically the approach I'd really rather not take :) and here's why:

You want to provide stability for you contributors so that if a problem is 
introduced, you don't halt development on that unrelated feature to diagnose 
the upstream issue.

You want to ensure that when users install your software, it works with 
everything that's currently released.

Ideally your CI configuration would be a timed job which builds against trunk 
and PR builds that build against a release, and then some process in place for 
reacting to regressions and filing a ticket upstream with us so we can work out 
what to do.

Easier said than done though, I know!  I wish we had something like this on 
Twisted for all of our dependencies, but we have had enough difficulty keeping 
our CI configuration current based on what cloud provider is falling over this 
month ;-).

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Richard van der Hoff]

On 03/03/2021 08:07, Glyph wrote:


If dependencies could start testing against Twisted trunk in some 
capacity, we could get notified close to when unintentionally breaking 
changes occur, and dependencies can let us know well before the 
release happens, and we can either revert or they can fix things if 
the error is on their end.  If initially, say, crossbar and matrix 
would like to work with us to set up some kind of repeatable pattern 
we can suggest to others, that would be great.


I'll take this to the Synapse team to discuss further, but we could 
probably easily arrange for one of our CI runs to install Twisted trunk 
from git instead of pypi, which might be a good start.



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Release blocker: Use latest pydoctor release ?

[Maarten ter Huurne]

On Sunday, 28 February 2021 23:04:49 CET Craig Rodrigues wrote:

> With respect to API docs, I am not as familiar with the whole process,
> with how they are generated
> and what are doing with readthedocs vs. docs on twistedmatrix.com.
> API docs are generated and don't live in the source tree.
> 
> For example:
> https://twistedmatrix.com/documents/current/api/twisted.html
> 
> Is the long term direction to get rid of that, and point everything at
> readthedocs?

Yes, as far as I know the intention is to reduce the amount of 
infrastructure that has to be maintained by Twisted developers.

Adi is doing the actual work for the migration; I only contribute 
indirectly by reviewing PRs that make pydoctor integrate better with 
Sphinx.

> Since you have done a lot of work in this area, can you shed some
> light on what you think the future direction of all this stuff should
> be with respect to the API docs?

There are efforts to make the output of pydoctor more user friendly. 
This is mainly done by Tristan, but I occasionally work on it as well. 
In the next major release we should have a clearer presentation of 
parameter types, more navigation links and, if it's ready in time, a 
side bar containing a page outline.


The main thing I'm working on is improving accuracy. Currently pydoctor 
can draw conclusions based on incomplete information if import cycles 
are present and with type annotations being added, there are more import 
cycles than before. To solve this, I want to separate the parsing and 
local analysis from the analysis that involves multiple documented 
objects, and run the latter only after all of the former is finished.

On Twisted's side of improving accuracy, we currently have a lot of 
'@type' fields in the docstrings which aren't correct. In particular, 
there is still a lot of fallout from the Python 2 to 3 migration, where 
a documented type of 'str' can either mean 'bytes' or 'str'. I think 
that most '@type' fields should be replaced by type annotations, in 
which case mypy will verify that the documented type matches the type 
that the code is actually using.

Something that might help here is to generate type stubs from the 
information found in docstrings and then automatically apply those stubs 
to Twisted's code. A possible implementation would be for pydoctor to 
export the result of docstring parsing as JSON and then a separate tool 
could generate type stubs from that data.


Twisted's customizations to pydoctor are a bit of a pain, since changes 
in pydoctor break Twisted's API docs quite often. The customizations 
exist in two parts: code and templates.

For the code customizations, these are done using subclassing, so the 
tight coupling makes it hard to change pydoctor's design without 
breaking the customizations. If possible, I'd prefer to remove the need 
for these customizations. If that isn't possible, we'd have to design a 
new plugin interface that is a lot more shielded from pydoctor's 
internals.

For anyone interested, the details are discussed here:
https://github.com/twisted/pydoctor/issues/315

For the template customizations, we're splitting up the templates into 
smaller chunks. This will eliminate or at least greatly reduce the 
amount of copy-pasted template content, which should allow Twisted to 
switch to new major pydoctor reeleases without having to sync the 
template content changes almost every time.

The PR for the template rework is here:
https://github.com/twisted/pydoctor/issues/299

Bye,
Maarten



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Jean-Paul Calderone]

On Wed, Mar 3, 2021 at 3:08 AM Glyph  wrote:

>
> On Mar 2, 2021, at 6:10 PM, Jean-Paul Calderone 
> wrote:
>
> Policy aside, this change doesn't seem like much of an improvement to me.
> If I were to guess, I would guess the change was made to satisfy some check
> Mypy is now being asked to make about Twisted.  If that's the case, it
> seems unfortunate that real-world software is suffering so that a synthetic
> goal can be achieved.  I do recognize there is a perception that practical
> value can come from attending to the errors Mypy reports.  It would
> probably benefit everyone if more care were taken to consider the
> real-world consequences of changes that are made to satisfy the
> non-real-world goalposts set by tools like Mypy.
>
>
> I do think that Mypy was likely highlighting a real issue here, and it
> should have been fixed.  We had documented interfaces already, and we were
> failing to adhere to them properly, and Mypy was complaining about that.
>
> For easy reference, the change that made these fixes is here
> https://github.com/twisted/twisted/pull/1298
>
> This genre of fix is definitely something we should unwind over time,
> although it does make it easier to start mypy-clean rather than have
> hundreds of exclusion lines that need to be manually adjusted, so on
> balance I agree with this tradeoff at the point it was made.
>

Broadly, I agree.  But not with this part.  It seems like there is clearly
a trade-off that is better for everyone.  The trade-off represented by
#1298:

   - Breaks application code without providing any new functionality or
   fixing any bugs
   - Captures a long list of TODOs as an arbitrarily complex collection of
   sources spread across the entire codebase
   - All the work of addressing the problems still remains to be done

 Contrast this with any basic ratchet-style approach (for example, capture
the list of mypy errors, then allow any PR that either removes some or
doesn't add any new ones):

   - At the outset, no application code breaks because nothing has actually
   changed
   - As work towards *fixing* the TODOs progresses, if it *does* break
   application code then at least applications have a chance at some benefit
   - As the work towards fixing the TODOs progresses, maintainers can
   easily look up what issues remain by consulting the list of errors that
   feeds the ratchet-style job.

The exact mechanism for the ratchet isn't the point here.  Maybe Mypy plays
more nicely with inline comments telling it to ignore something, I don't
know.  The point is:

   - A centralized list of stuff left to do is better than a mess smeared
   across the whole codebase
   - If we're going to risk breaking applications we should at least try to
   offer some value to them
   - We shouldn't make applications deal with every change twice when we
   could just disturb them once

Jean-Paul


> Specifically what I mean by "this genre of fix" is that you can always
> make mypy happy by transforming code like this:
>
> class Thing:
> pass
>
> Thing().stuff()
>
>
> into code like this:
>
> class Thing:
> @property
> def stuff(self):
> raise AttributeError("no such thing as `stuff`")
>
> Thing().stuff()
>
>
> The runtime behavior is the same, but now Mypy can't tell you about this
> class of bug any more.
>
> So, at some point, we should slowly unwind all `NotImplementedError`
> methods and find ways to either implement them for real or make their
> required use raise at import time.
>
> Finally: let's not develop an aversion to new tooling and change because
> it might create problems; experience over the last few years has shown me
> that Mypy can catch *tons* of real bugs and it's well worth getting the
> codebase to type check.  If we want to prevent breakages like this in the
> future, the answer is not to stop trying to get linters and typecheckers to
> run cleanly with arbitrary changes, but to figure out some kind of *continuous
> integration *solution that's actually continuous with our downstream
> dependencies
>
> If dependencies could start testing against Twisted trunk in some
> capacity, we could get notified close to when unintentionally breaking
> changes occur, and dependencies can let us know well before the release
> happens, and we can either revert or they can fix things if the error is on
> their end.  If initially, say, crossbar and matrix would like to work with
> us to set up some kind of repeatable pattern we can suggest to others, that
> would be great.
>
> -g
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

[Glyph]

> On Mar 2, 2021, at 6:10 PM, Jean-Paul Calderone  
> wrote:
> 
> Policy aside, this change doesn't seem like much of an improvement to me.  If 
> I were to guess, I would guess the change was made to satisfy some check Mypy 
> is now being asked to make about Twisted.  If that's the case, it seems 
> unfortunate that real-world software is suffering so that a synthetic goal 
> can be achieved.  I do recognize there is a perception that practical value 
> can come from attending to the errors Mypy reports.  It would probably 
> benefit everyone if more care were taken to consider the real-world 
> consequences of changes that are made to satisfy the non-real-world goalposts 
> set by tools like Mypy.

I do think that Mypy was likely highlighting a real issue here, and it should 
have been fixed.  We had documented interfaces already, and we were failing to 
adhere to them properly, and Mypy was complaining about that.

For easy reference, the change that made these fixes is here 
https://github.com/twisted/twisted/pull/1298 


This genre of fix is definitely something we should unwind over time, although 
it does make it easier to start mypy-clean rather than have hundreds of 
exclusion lines that need to be manually adjusted, so on balance I agree with 
this tradeoff at the point it was made.

Specifically what I mean by "this genre of fix" is that you can always make 
mypy happy by transforming code like this:

class Thing:
pass

Thing().stuff()

into code like this:

class Thing:
@property
def stuff(self):
raise AttributeError("no such thing as `stuff`")

Thing().stuff()

The runtime behavior is the same, but now Mypy can't tell you about this class 
of bug any more.

So, at some point, we should slowly unwind all `NotImplementedError` methods 
and find ways to either implement them for real or make their required use 
raise at import time.

Finally: let's not develop an aversion to new tooling and change because it 
might create problems; experience over the last few years has shown me that 
Mypy can catch tons of real bugs and it's well worth getting the codebase to 
type check.  If we want to prevent breakages like this in the future, the 
answer is not to stop trying to get linters and typecheckers to run cleanly 
with arbitrary changes, but to figure out some kind of continuous integration 
solution that's actually continuous with our downstream dependencies

If dependencies could start testing against Twisted trunk in some capacity, we 
could get notified close to when unintentionally breaking changes occur, and 
dependencies can let us know well before the release happens, and we can either 
revert or they can fix things if the error is on their end.  If initially, say, 
crossbar and matrix would like to work with us to set up some kind of 
repeatable pattern we can suggest to others, that would be great.

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python