Re: [Twisted-Python] Enable pre-commit.ci for twisted/twisted

[Hynek Schlawack]

>> I am +0 on this change due to security reasons...but I do think that it will 
>> reduce a bit of the frustration for first time contributors.
>> 
> 
> I'm +1; we've accepted a lot of cloud risk already, and this seems like a lot 
> of benefit for minimal additional risk.  Is this asottile running the show?  
> The website is not entirely clear.

He is. I’ve had it enabled on both structlog & argon2-cffi for a while now FWIW.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Post Release updates

[Hynek Schlawack]

> On 28. Feb 2021, at 21:42, Adi Roiban  wrote:
> 
> And we have not yet decided if the trunk should be using `.dev` or .post` :(
> 
> https://twistedmatrix.com/trac/ticket/9542
> 
> I am +1 for .post1  ... as it has the semantic of a post release version.
> while .dev0 is a pre-release.

FWIW, I think it’s common (aka _I_ do it that way) that .post are indeed 
post-releases like a documentation update that doesn’t warrant a proper update. 
PEP 440 specifies it and RTD has good support for that. E.g. 
https://www.attrs.org/en/17.3.0.post2/

IIRC Twisted doesn’t bump before the release, which makes this whole workflow 
unfeasible? But it’s something to keep in mind when deciding.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] reactor for Linux io_uring

[Hynek Schlawack]

>> What's more interesting is that io_uring accepts files as 
>> well as network/pipe handles: avoiding the need for threads.
> 
> What threads? Why do you call out file FDs different from socket FDs?
> 
> The point of io_uring is to avoid transitions between user and kernel right?
> Nothing to do with thread.
> 
> In current twisted you can run complex network code without threads
> already.

Because before io_uring there was no reliable way on Linux to do async file I/O 
and everybody is forced to use thread pools for it.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Hynek Schlawack]

> On 14. May 2020, at 07:04, Amber Brown (hawkowl)  
> wrote:
> 
> - MacOS doesn't ship a Python 3, but homebrew/python.org offer 3.8 easily.

FWIW this is not true anymore. Catalina comes – for all its faults – with 
Python 3.7:

❯ /usr/bin/python2

WARNING: Python 2.7 is not recommended.
This version is included in macOS for compatibility with legacy software.
Future versions of macOS will not include Python 2.7.
Instead, it is recommended that you transition to using 'python3' from within 
Terminal.

Python 2.7.16 (default, Feb 29 2020, 01:55:37)
[GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.29.20) (-macos10.15-objc- 
on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D

~ took 6s
❯ /usr/bin/python3
Python 3.7.3 (default, Apr  7 2020, 14:06:47)
[Clang 11.0.3 (clang-1103.0.32.59)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] farewell to twisted.news

[Hynek Schlawack]

> If you love NNTP, and would like to help this package stick around by 
> maintaining it, that would be great!  However, please note that at this point 
> it's not worth discussing unless you have the actual free time to dedicate to 
> doing the py3 port, cleaning up the test coverage, and generally keeping it 
> updated and maintained going forward.  Any PR wanting to un-deprecate it is 
> also going to have to get it working on py3 :-).

I don’t think this is necessarily the narrative that ought to be had in 2019.

I’d push for: if you love NNTP and want it to stick around, do all the work, 
put it on PyPI, and if it’s good, we’ll move it into the Twisted orga on 
GitHub.  NNTP support looks quite independent to me and at this point we 
can/should apply Amber’s core summit talk to Twisted’s batteries just as well..

Cheers,
—h

P.S. Please don’t storm out of the room. ;)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] pypy3 builds passing?

[Hynek Schlawack]

> It does on master, i think, but unsupported doesn't run on PRs.
> 
> I'm happy to move it to supported if we're happy with it! I think since it 
> runs all the main reactors plus asyncio, we can be relatively trusting of it 
> being green now...
> 
> 
> That build runs select, poll, epoll, and asyncio reactors.  So since there 
> are about 12,000 tests, running 4 times, that is about 48,000 tests
> for this builder.  That took 39 minutes.

Does it run under Coverage? attrs’ test suite took almost 20 minutes when 
running under PyPy3 and coverage so we switched it off for everything except 
py27 and py36.___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] (no subject)

[Hynek Schlawack]

Hi Justin,

> How can I help you with Twisted?  How can you help me move over the cusp into 
> active, useful contribution?

Have you seen 
>?

Cheers
—h___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted and Flask or Django

[Hynek Schlawack]

>> I have to disagree here BTW.  When I started using Twisted as a WSGI 
>> container, I was vexed by its lack of docs (and I’m still confused about 
>> many details).  There’s a few blog posts around but none of them reach the 
>> straight-forwardibility of e.g. gunicorn’s docs.  I’ve ended up asking 
>> around on IRC, stealing from Donald/PyPI, and reading source code.
> All I was saying here is that frameworks have gotten better about being "just 
> WSGI" and not requiring a lot of crazy customization on the framework side of 
> things.  But, you're absolutely right that we need better docs, many parts of 
> the setup are not obvious!

Yeah, I didn’t actually mean to disagree as much as use it as an opportunity to 
voice something that has been brewing in me for a while. :)

>> There’s also barely any information on *why* one would want to use Twisted 
>> as a WSGI container.  And please keep in mind that to many, many (most 
>> likely: the majority) of Python users “pure Python” doesn’t mean “safe” but 
>> “slow.” C.f. all those “lightning fast pure C” frameworks popping up left 
>> and right, upvoted to nirvana on reddit and its likes.   If we want to do 
>> marketing, we have to be a tad more user-focused. :)
> Yep.  And things like Moshe's talk are wonderful resources to try to 
> illuminate this unfortunately dark area, so I do appreciate people taking the 
> time to do them.  I do also think that being a bit more strategic about it 
> might result in vastly greater adoption though.

Yeah Moshe’s evangelism made me interested in the topic but it’s hard to find 
anything more than his material.  Particularly a good reference guide.  As you 
wrote, the particular integration with web frameworks is trivial (but should be 
provided!)

>> What I’m saying is that this topic shouldn’t be relegated into some wiki 
>> (where information famously is going to die) but should be a prominent tab 
>> on the landing page (maybe with additional information for specific 
>> frameworks in the wiki).
> The front page is also the wiki :-).  I'm happy to have it there, I just 
> think that it should perhaps be workshopped a little bit on a draft page 
> first.

Heh absolutely.  Just wanna stress that it should come right after “Web Server” 
in the “Code Examples” tabs. :)

***

Oh and I’d like to propose to officially merge the twisted-web mailing list 
into this one.  I don’t think it makes any sense to separate those two anymore. 
 It’s hard to do anything without web nowadays. :)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted and Flask or Django

[Hynek Schlawack]

> For the most part, general WSGI documentation should get you most of the way 
> there with specific frameworks though.  This used to be a lot worse but in 
> recent years there's been a lot of convergence.

I have to disagree here BTW.  When I started using Twisted as a WSGI container, 
I was vexed by its lack of docs (and I’m still confused about many details).  
There’s a few blog posts around but none of them reach the 
straight-forwardibility of e.g. gunicorn’s docs.  I’ve ended up asking around 
on IRC, stealing from Donald/PyPI, and reading source code.

I honestly believe, we’re leaving money on the table here.  Using Twisted as a 
WSGI container should be first class use case on the web page since it presents 
a nice entry drug into the ecosystem.

There’s also barely any information on *why* one would want to use Twisted as a 
WSGI container.  And please keep in mind that to many, many (most likely: the 
majority) of Python users “pure Python” doesn’t mean “safe” but “slow.” C.f. 
all those “lightning fast pure C” frameworks popping up left and right, upvoted 
to nirvana on reddit and its likes.   If we want to do marketing, we have to be 
a tad more user-focused. :)

***

What I’m saying is that this topic shouldn’t be relegated into some wiki (where 
information famously is going to die) but should be a prominent tab on the 
landing page (maybe with additional information for specific frameworks in the 
wiki).

Cheers,
—hs
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

>> Are you talking about building Docker containers on the fly? 
> 
> I’m a bit baffled what gave you that idea after I’ve spent days arguing for 
> strict build/runtime separation?
> 
> I was just trying to figure out where you were having this problem:
> 
> > ZFS and apt-get + lots of files in a deb = omgiwannamurdersomeone
> 
> I heard the opposite of what you meant by this comment -- that you were 
> deploying apt-gets + debs using docker and the combination had you wanting to 
> murder someone.

It was not clear to me you were referring to that line in particular.  
Especially because you were talking about Docker and not about debs.

The problem is a long-standing “bug” (scare quotes because it’s more of a 
performance regression) in ZFS-on-Linux: 
https://github.com/zfsonlinux/zfs/issues/3857 
 (I believe there are more 
related tickets but I don’t have the time to investigate).  It’s related to 
excessive sync calls for every single file that accumulate very fast if you put 
a virtualenv with many files into a deb.  Basically I have to run containers 
with Python debs in YOLO mode (sync=False) if I don’t want installs to take 
more than a minute while it takes almost no time on ext4 or XFS.

That said, our Docker containers run on ext4 because our Docker daemon runs in 
an LXC and you cannot access the pool from within containers (yet, as I’ve 
hear).  Which makes it run in VFS mode which is even slower (I’ve measured 
factor 3x vs overlay2 in our CI) because it means it has to create a physical 
copy of each layer.

I suppose we were talking about different things because I don’t quite 
understand why you’re upset.___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

> Are you talking about building Docker containers on the fly? 

I’m a bit baffled what gave you that idea after I’ve spent days arguing for 
strict build/runtime separation?

> We use Docker extensively, but our build machine makes images that we push to 
> Dockerhub (private repos).  This has a lot of advantages:
> Our images (on the hub) are effectively pinned at the version they were built
> Our test and production servers (can, if we want) always get exactly the same 
> image (even if we need to rebuild a server months later)
> We test all our servers so we only have to manually pin packages (python or 
> apt) if we run into regressions or other incompatibilities (i.e. an upgraded 
> package that is no longer compatible with a manually pinned package)
> Our build machine caches all the intermediate images (i.e. after each docker 
> step).  We intentionally sequence our images to place oft-changing items at 
> the end.  
> Unless I change the list of apt packages, that layer is never rebuilt.
> We have an extra step that uploads *just* the requirements files before pip 
> installing
> Our last step is the app code so changes to this layer are just a cached 
> layer + PUT (i.e. seconds)
> This optimization also makes our containers super efficient to upgrade 
> because we only download the changed layers
> This sounds like it covers a lot of the PEX advantages plus the added 
> benefits of containerization.

I don’t see anything that contradicts anything that I (or glyph) have written.  
At this point we were merely discussing what kind of isolated build artifact 
goes into the container/deb: a pex (= single file) or a vanilla venv (= 
directory structure).

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

> ...  Contrary I’m not a super fan of having one opaque blob on server; the 
> transparent structure of a virtualenv is something I learned to appreciate. 
> ... 
> 
> A zipfile containing a virtualenv (pex) isn't all that opaque -- with the 
> right editor plugins, you can even *cough* edit the contained source inside 
> one. 
> 
> Back when pex was a lot younger, this was handy for quick debugging as we 
> were migrating from source-in-venvs-on-bare-metal-chroots to 
> pexed-in-mesos-containers

To be clear: I’m not trying to talk anybody out of using pex. :)  I’m just 
saying there’s a slight downside (+ another tool) and no tangible upsides in 
Docker for me so it’s not a good fit for me.

That said, I’m gonna try it out in my deb-based deployments (contrary to 
popular believe, you can’t/shouldn’t put *everything* into containers) because 
we run ZFS and apt-get + lots of files in a deb = omgiwannamurdersomeone.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

> 
>> I'm trying to point out that *some* might find the system Python attractive 
>> because they can use OS-supplied python packages in lieu of the effort of 
>> setting up a binary build infrastructure.
> 
> So, I think everyone has a different perspective here; Hynek seems to be 
> saying "all containers all the time" ;-).  But from my perspective, linux 
> distros provide a super useful service by providing a gigantic integrated 
> build environment for TONS of C code.  Avoiding the time and cost associated 
> with setting up, i.e., an ImageMagick dev environment is _WELL_ worth the 
> complexity of distro packages.  Multiply this out by dozens of dependencies 
> that a large application or site ends up needing, and distro packaging pays 
> for itself many times over.

I’m not sure what you mean by me meaning “all containers all the time” I feel I 
should rephrase my point:

- You should statically link your Python application so you have always full 
control over your deployments.
- Don’t ship build tools to prod servers.

The way we can achieve that is by building virtualenvs on build servers.  *How* 
you get those virtualenvs to the deployments target I don’t care.  Whether you 
pex them or not, I don’t care either¹.  I’ve used .debs with great success in 
the past and I’m using Docker containers now (and so far we’re not out of 
business :)).

I consider Linux distributions very nice build environments for my own 
software. :)  And there’s thing we totally don’t compile ourselves like 
OpenSSL.  It just doesn’t make any goddam sense to rely on the limited and 
outdated offer of Python modules from your distribution.  Even if they don’t 
upgrade them under your butt and introduce regressions.

—h

¹ For me apps always go to the same location so relocability is not an issue.  
Contrary I’m not a super fan of having one opaque blob on server; the 
transparent structure of a virtualenv is something I learned to appreciate.___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

> As a very simple example: if you have a traditional (non-container) Linux 
> system hosting a Python application in a virtualenv, and you deploy a Python 
> app to a virtualenv e.g. using Puppet or Ansible, you either need to:
> 
> 1. Use no C extensions
> 2. Hope there's a manylinux1 binary wheel
> 3. Use the OS package and --system-site-packages
> 4. Compile the C extensions and make them available to pip
> 
> #2 seems useful now that I know about it but - correct me if I'm wrong - the 
> manylinux1 permitted C dependencies are super-tiny, and would not permit e.g. 
> cryptography or psycopg2?
> 
> #4 is what you are advocating for I believe? But can we agree that for 
> smaller projects, that might seem like a lot of repeated work if the package 
> is already available in the OS repos?

You always say “repeated work” but I hear “small Python script”. :)  We have a 
few Python projects (not written/maintained by me :)) that did rely on system 
packages “because it’s easier”.  Guess what!  The other day Ubuntu upgraded 
PyCrypto (needed Ubuntu Trusty’s ancient Paramiko) under our butts, introduced 
a regressing and everything blew up: https://www.ubuntu.com/usn/usn-3199-2/ 
 . That particular project is a 
virtualenv now too.

The Python build chain got so good on Linux by now that I really don’t even 
think about C extensions anymore.  Windows is a different story entirely of 
course. :|

> I'm trying to explain why, based on the efforts we expend locally, it could 
> seem attractive to smaller sites.

Honestly, in the years I’ve been running Python services of different sizes, I 
have found that distro-provided system packages – unless you are writing 
software *for* a distribution – are loaded with so many downsides that they’re 
almost never worth it.  They’re a shortcut and shortcuts usually bite back 
*eventually*.

—h___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

 I don’t see how that’s tedious since a compute does that for me.
 Although I don’t see any value at wheeling them (and some packages
 cannot be wheeled); my CI builds a venv and puts it into a container.
 There’s nothing tedious about it at all.
>>> I find the idea of running throwaway environments to generate a big blob of 
>>> tarball'd python+libs, then copying said tarball to actual containers, a 
>>> rather retrograde step by comparison with established package/build 
>>> infrastructure tools.
>> 
>> I have to disagree here:  I don’t want build tools of any kind in my final 
>> containers therefore I build my artifacts separately no matter what 
>> language.  Of course you can just build the venv on your build server 
>> without wheeling up a temporary container and then package it using Docker 
>> or DEB or whatever.  You should be separating building and running anyway so 
>> Python – as much as I’d like Go-style single binaries too – is in no way 
>> special here.  The nice thing about temporary containers though is that I 
>> can do all of that on my Mac.
> 
> It's worth pointing out that if you don't want a Go build toolchain in your 
> container, you have exactly the same problem.  

I thought I pointed that out in my very first sentence. :D

What I don't quite understand is how people can be in love with Go’s static 
linking but complaining about Virtualenvs in deployments.   Unwieldy as 
virtualenvs are: *for Python code* they are exactly that: statically linked 
build artifacts.  The principles are very similar, the execution is arguably 
better for Go.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

>> I don’t see how that’s tedious since a compute does that for me.
>> Although I don’t see any value at wheeling them (and some packages
>> cannot be wheeled); my CI builds a venv and puts it into a container.
>> There’s nothing tedious about it at all.
> I find the idea of running throwaway environments to generate a big blob of 
> tarball'd python+libs, then copying said tarball to actual containers, a 
> rather retrograde step by comparison with established package/build 
> infrastructure tools.

I have to disagree here:  I don’t want build tools of any kind in my final 
containers therefore I build my artifacts separately no matter what language.  
Of course you can just build the venv on your build server without wheeling up 
a temporary container and then package it using Docker or DEB or whatever.  You 
should be separating building and running anyway so Python – as much as I’d 
like Go-style single binaries too – is in no way special here.  The nice thing 
about temporary containers though is that I can do all of that on my Mac.

—h
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

>>> So my problem is that while my protocol stack spins up very nicely and 
>>> trivially easy using the twistd daemon, the standard python3-twisted 
>>> package for 14.04 is wy behind and doesn't include twistd (!), and I 
>>> haven't found a good backport.  I'd also like to avoid doing things that 
>>> make life hard for DevOps, such as pip3 installing and risking a package 
>>> manager fight.
>> 
>> The general solution to this problem is "don't use the system Python 
>> environment to deploy your applications".
>> 
>> This is what virtualenv is for.
> 
> Yes, well, the standard practice around here is to wrap up deliverables in a 
> .deb so that the DevOps tools can do automated deploys.  I suppose it is 
> possible to wrap up a venv in a .deb, but I’ve never done looked into it. 
> Pointers welcome.

https://github.com/spotify/dh-virtualenv 
 and 
https://hynek.me/articles/python-app-deployment-with-native-packages/ 
 come to 
mind.

N.B. the article is by me and I’m currently migrating to Docker/Nomad.  But it 
served us very well for many years.___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

[Hynek Schlawack]

>> I'm tempted to launch into a diatribe about namespacing, containers, and
>> application isolation generally, but before I do - why is it that you
>> /want/ to use the system Python environment?  Had you just not
>> considered the option of virtual environments?
> Awesome though it is, virtualenv can be very tedious if you need to install 
> hundreds of megabytes of compiler and -devel packages. System packages are 
> attractive precisely because you can avoid this.

That’s why you should use a build server and not ship build environments.

> I've had to do all sorts of tedious things with containers where I spin up a 
> temporary container to build a bunch of .whl files, then actually install 
> them in the final container - all to avoid bloating the container image with 
> the build tools.

I don’t see how that’s tedious since a compute does that for me.  Although I 
don’t see any value at wheeling them (and some packages cannot be wheeled); my 
CI builds a venv and puts it into a container.  There’s nothing tedious about 
it at all.

> It's a real shame that binary wheels on Linux/PyPI aren't a thing.

This is incorrect.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] trial cant find my class, example from twisted docs

[Hynek Schlawack]

>>> any idea why this does not work?
>> 
>> Most likely you are being bitten by the recent change to not add '.'
>> to sys.path automatically.
>> 
>> Does `PYTHONPATH=. trial ...` run your code as expected?
>> 
> Yes, thanks very much. this kind of change really makes one doubt ones
> abilities.
> 
> Should i write a ticket for this?

No, you should start installing your packages and  applications in dev lest 
packaging errors come back in prod and bite you. :)

More on that: https://hynek.me/articles/testing-packaging/___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Removing support for old Python 3 versions

[Hynek Schlawack]

In general I'd say we should take into account LTS releases. 

OTOH Python 3 adoption is still rather low and tends toward newer versions with 
fast upgrading.

There's deadsnakes and there's SC and there's Docker.  I think it's totally 
fair to drop everything before 3.5 since that gives us nice goodies that make a 
lot of sense for Twisted.

A volunteer driven project gets to choose their own priorities and there's no 
entitlement to shiny new features on old Pythons. 

3.3 should die immediately.  3.4 should follow ASAP.

Sent from my phone apologies for top posting.

> Am 20.02.2017 um 22:46 schrieb Amber Brown :
> 
> Hi everyone!
> 
> Currently, Twisted supports five Python versions. These are 2.7, 3.3, 3.4, 
> 3.5, and now 3.6. This is more Python versions than Twisted has ever really 
> supported in the past, and even though it is great to support a lot of 
> runtimes, it is becoming a greater burden on Twisted to support so many.
> 
> The major reasons for this is that newer Python 3 versions include useful 
> features (such as % formatting for bytes) that earlier ones do not. This 
> leads us to having two, or sometimes three, ways of doing something - the 
> Python 2 way, the early Python 3 way, and the new Python 3 way. This 
> introduces complexity (and bugs!) and means we can't really take advantage of 
> anything nice Python 3 gives us in the future, as we still remain compatible 
> with 3.3 and 3.4.
> 
> As such, I would like to remove support for these Python versions. Using 
> recent statistics (https://langui.sh/2016/12/09/data-driven-decisions/) we 
> can infer that Python 3.3 is on the whole uncommon, despite it being the 
> default Python 3 for Ubuntu 14.04 LTS. We can further infer that adoption of 
> new major versions of Python 3 (Python 3.5 was barely a year old by the time 
> of these stats and yet dwarfed 3.4) is rapid, and so it seems only the latest 
> two or so are worth supporting.
> 
> The other rationale I have for this is that we no longer have the problem of 
> CentOS 6-style Python 2.6 being put into new production because it's what 
> people ran -- the Brave New World of Docker and containerisation puts us in a 
> different position than we were with Python 2.6. Although this is purely 
> anecdotal, it appears lots of Python 3 installations are not through the 
> operating system (due to how slow the cycles are, and the proliferation of 
> PPAs and Docker containers), and therefore being tied to whatever current OSs 
> support is not doing any Python 3 users a real service.
> 
> With this rationale, I propose that the next version of Twisted (most likely 
> 17.3) will be the last to support Python 3.3. The question of Python 3.4 
> support is something that will need further discussion (where 3.3 just plain 
> has the writing on the wall), but I believe a removal of support not long 
> after is reasonable, considering the 3.4 installed base is smaller than that 
> of 2.6. 
> 
> Thoughts?
> 
> - Amber Brown, Release Owl
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Waiting time for tests running on Travis CI and Buildbot

[Hynek Schlawack]

> We can drop support for 3.3 of course, but that's a separate discussion.

+1 since 3.3 wasn’t ever part of any LTS distribution.  3.4 will live for a 
while but it’s also the first really good Python 3. :)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Challenge Coins!

[Hynek Schlawack]

>> You might be eligible to receive one, but first we need enough committers to 
>> buy them in sufficient quantity that they can exist at all :).
> 
> Heh, let’s assume I’m an idiot and didn’t extract that message from the 
> original email.
> 
> In that case, I’m willing to take 5 coins as well. I can also provide 
> US-based forwarding addresses if needed.

Guess I’d take 5 too…I don‘t have a fwd address (save some PyCon mules :)) but 
maybe you could send it via Cory or something?



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] overview: new review queue venue

[Hynek Schlawack]

Ah finally a fine bike shedding thread that gets everyone involved. ;)

> Right now, we need to manually vet each change before sending it to 
> buildbots, because they are shared mutable environments that we can't afford 
> to have running untrusted code automatically.  If we could switch to Travis 
> and Appveyor, then we could let them worry about malicious code, which would 
> allow contributors to get instant feedback, rather than waiting for reviewers 
> to manually run the builders.

I have two points to add:

1. Appveyor is terribly slow and sometimes a bit flaky.  I use it for 
argon2_cffi’s wheels and it drives me bonkers.  It should never become an 
essential part of anything.  As a first line of defense it’s fine of course.
2. PyCA has a workflow for Jenkins & GitHub by telling a bot to vet changes.  
You can see it here in action: 
https://github.com/pyca/cryptography/pull/2914#issuecomment-220592167 AFAIK 
that’s been mostly Paul’s work.  Aren’t you kind of his boss or something *hint 
hint*? ;)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] overview: new review queue venue

[Hynek Schlawack]

>>> A lot of projects do follow this workflow, and maybe it will be fine for 
>>> us.  The real question is; is FreeBSD support really worth it for the cost 
>>> to contributors, since that's the only platform we currently support but 
>>> can't test?
>> 
>> I'm guessing that we have more FreeBSD users than Windows users ;)
> 
> 
> I realize it can feel like that sometimes, but Google Analytics suggests the 
> large majority of our visitors (45%) are on Windows.  By contrast, 0.05% are 
> on FreeBSD.  Granted, that's a very high percentage of FreeBSD clients for 
> the Internet at large, but nevertheless, I think your perspective may be 
> slightly statistically skewed.

I don’t think there’s a meaningful correlation between what people use to 
browse docs vs. what people run Twisted on.

E.g. Netflix & WhatsApp run on FreeBSD; do you think their tech staff has 
FreeBSD on their desktops? :)

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] [ANN] prometheus_async: asyncio/Twisted-aware Python Prometheus instrumentation

[Hynek Schlawack]

Dear fellow friends of asynchronous software,

maybe some of you have already bumped into the Prometheus monitoring system 
 and liked it like I do (in any case, I’d like to invite 
you to my PyCon US talk on that topic: 
!)

And while it’s great that Python is a first class citizen due to the official 
Python client library , asyncio 
and Twisted sadly aren’t!

That’s why I’ve just released prometheus_async: 
https://prometheus-async.readthedocs.io/

First and foremost it wraps the metrics from the official client (you don’t 
want *me* to do math!) and makes them work properly on coroutines and Deferreds 
(and makes them well-behaved decorators too but that’s a topic for another 
day…).

Additionally, it adds a few goodies:

- Metric-exposure via aiohttp that ist much more flexible than what comes with 
the stdlib-based official solution.
- …that can also be started in a separate thread.  That means you can use them 
in regular, *synchronous* Python 3 applications as well (I instrument all my 
Pyramid apps like that).
- Integration with service discovery.  Listen on port 0 and leave registration 
to Consul Agent (integration is pluggable, just implement two methods)!

Sadly the goodies are asyncio-only so far.  Partly because the official client 
has some Twisted Web support merged but not released yet.  Contributions are 
very welcome!

Cheers,
Hynek
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] pyOpenSSL 16.0.0 released

[Hynek Schlawack]

On behalf of PyCA – the Python Cryptography Authority – I’m anxious to announce 
that after almost a year since the 0.15.1 release of pyOpenSSL we’ve released 
the brand new 16.0.0.

A few organizational notes:

1. The pyopenssl-users mailing list and #pyopenssl IRC channel are deprecated.  
Please use cryptography-dev 
<https://mail.python.org/pipermail/cryptography-dev/> and #cryptography-dev on 
Freenode where you’re much more likely to get help.
2. The version scheme switched to CalVer because a 0.x version for a 15 years 
old project is rather odd and calling it 1.0 although we don’t expect a 2.0 to 
ever happen didn’t make any sense.  pyOpenSSL is a long-running project with 
strict backward-compatibility requirements and is hence better served with a 
calendar-based version scheme.
3. Please note that some of us will be doing a TLS/HTTPS workshop at PyCon US 
2016 so if you always wanted to learn about these things first hand, make sure 
to sign up: <https://us.pycon.org/2016/schedule/presentation/1786/>.  We've 
opted to receive no compensation and asked the organizers to send them to 
PyLadies instead.  So you’ll be doing good while learning something!

***

Release details:

While the list of changes looks short, a lot internal work happened:

72 files changed, 15511 insertions(+), 15063 deletions(-)

We’ve done our best to not break any existing applications; including by making 
the urllib3 and Twisted test suites part of our CI.

The full changelog can be found at 
<https://pyopenssl.readthedocs.org/en/stable/changelog.html>.

This is the first release under full stewardship of PyCA. We have made many 
changes to make local development more pleasing. The test suite now passes both 
on Linux and OS X with OpenSSL 0.9.8, 1.0.1, and 1.0.2. It has been moved to 
py.test, all CI test runs are part of tox and the source code has been made 
fully flake8 compliant.

We hope to have lowered the barrier for contributions significantly but are 
open to hear about any remaining frustrations.


Backward-incompatible changes:

• Python 3.2 support has been dropped. It never had significant real world 
usage and has been dropped by our main dependency cryptography. Affected users 
should upgrade to Python 3.3 or later.


Deprecations:

• The support for EGD has been removed. The only affected function 
OpenSSL.rand.egd() now uses os.urandom() to seed the internal PRNG instead. 
Please see pyca/cryptography#1636 for more background information on this 
decision. In accordance with our backward compatibility policy 
OpenSSL.rand.egd() will be removed no sooner than a year from the release of 
16.0.0.
Please note that you should use urandom for all your secure random number needs.
• Python 2.6 support has been deprecated. Our main dependency 
cryptography deprecated 2.6 in version 0.9 (2015-05-14) with no time table for 
actually dropping it. pyOpenSSL will drop Python 2.6 support once cryptography 
does.


Changes:

• Fixed OpenSSL.SSL.Context.set_session_id, OpenSSL.SSL.Connection.renegotiate, 
OpenSSL.SSL.Connection.renegotiate_pending, and 
OpenSSL.SSL.Context.load_client_ca. They were lacking an implementation since 
0.14. #422
• Fixed segmentation fault when using keys larger than 4096-bit to sign data. 
#428
• Fixed AttributeError when OpenSSL.SSL.Connection.get_app_data() was called 
before setting any app data. #304
• Added OpenSSL.crypto.dump_publickey() to dump OpenSSL.crypto.PKey objects 
that represent public keys, and OpenSSL.crypto.load_publickey() to load such 
objects from serialized representations. #382
• Added OpenSSL.crypto.dump_crl() to dump a certificate revocation list out to 
a string buffer. #368
• Added OpenSSL.SSL.Connection.get_state_string() using the OpenSSL binding 
state_string_long. #358
• Added support for the socket.MSG_PEEK flag to OpenSSL.SSL.Connection.recv() 
and OpenSSL.SSL.Connection.recv_into(). #294
• Added OpenSSL.SSL.Connection.get_protocol_version() and 
OpenSSL.SSL.Connection.get_protocol_version_name(). #244
• Switched to utf8string mask by default. OpenSSL formerly defaulted to a 
T61String if there were UTF-8 characters present. This was changed to default 
to UTF8String in the config around 2005, but the actual code didn't change it 
until late last year. This will default us to the setting that actually works. 
To revert this you can call 
OpenSSL.crypto._lib.ASN1_STRING_set_default_mask_asc(b"default"). #234

***

For PyCA,
Hynek Schlawack


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Auto-Re: Weekly Bug Summary

[Hynek Schlawack]

I wonder, is something weird with our Mailman configuration?  Because I haven't 
seen any other list (some with many more subscribers) that has this regular 
auto-responder problem.

Sent from my phone.

 Am 30.08.2015 um 08:10 schrieb wangl...@dhcc.com.cn:
 
 您的邮件已收到，我会尽快处理，谢谢！
 
 
 --
 Best Regards,
 王立宇
 --
 DHC Software Co.,Ltd.
 东华软件股份公司
 Add：11th floor,DHC Office Building,ZiJin Digital Park ,Zhongguancun,Haidian 
 District, BeiJing ,China
 北京市海淀区中关村紫金数码园3号楼东华合创大厦11层
 邮编 100190   手机：18612695237
 
 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Auto-Re: Weekly Bug Summary

[Hynek Schlawack]

This isn't a new problem either; it's just more prevalent this year.  Typically 
emerges around some kind of holidays. 

Sent from my phone.

 Am 30.08.2015 um 09:01 schrieb Glyph gl...@twistedmatrix.com:
 
 
 
 On Aug 29, 2015, at 11:38 PM, Hynek Schlawack h...@ox.cx wrote:
 
 I wonder, is something weird with our Mailman configuration?  Because I 
 haven't seen any other list (some with many more subscribers) that has this 
 regular auto-responder problem.
 
 Nothing's changed about our configuration, so I'm not sure what has happened. 
  Did some other mail server change how it did auto-responses recently?
 
 -glyph
 
 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Considerations for Twisted prereleases on PyPI

[Hynek Schlawack]

 So, the question is, do people think we should start putting them on PyPI? Is 
 it worth some users on ancient pips inadvertantly getting (admittedly quite 
 stable) prereleases? Would you use it, and would you be more likely to test 
 Twisted prereleases if they were distributed like this (in addition to the 
 tarball)?

Yes, I would and yes we should.  Current (more than 1 year old) Ubuntu LTS 
comes with pip 1.5.4, CentOS 7 comes with virtualenv 1.10.1 which seems 
(https://pypi.python.org/pypi/virtualenv/1.10.1) to come with pip 1.4.1.


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Proposal -- Code of Conduct

[Hynek Schlawack]

 I am sure everyone understands that the Twisted community would love more 
 diversity. While it is hard to achieve, it should be easy to remove one of 
 the obvious blockers -- making underrepresented groups feel more welcome. 
 Thanks for taking this on, Moshe.

+1

 My current draft, including instructions on how to build it, is in 
 https://github.com/moshez/twisted-coc 
 https://github.com/moshez/twisted-coc . I have intentionally not made the 
 built documents available, in an attempt to avoid someone picking them up 
 before they're approved by us. 
 
 Why isn't this repository either (A) just a simple text file saying we have 
 adopted the Django CoC or (B) a very small fork of something else?  One of 
 the concerns is licensing; if the text comes via Django, Django credits the 
 Speak Up! project, which is CC-BY, apparently from this repository: 
 https://github.com/jnoller/talk-mentorship 
 https://github.com/jnoller/talk-mentorship.  Another is... is Twisted 
 really distinct enough to need its own CoC?  Just s/Django/Twisted might be 
 good enough?  (Since this is not a fork, figuring out if anything else has 
 changed is rather tedious, even after having read both ;)).

I wonder whether it might make sense to just say we adopt 
https://www.python.org/psf/codeofconduct/ 
https://www.python.org/psf/codeofconduct/ ?

What I would really love is if we could have our own diversity statement like 
Django has: https://www.djangoproject.com/diversity/ 
https://www.djangoproject.com/diversity/

Cheers,
—h

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] pypy support

[Hynek Schlawack]

On 7 Jun 2015, at 7:25, Glyph wrote:

Many Twisted core developers (myself included) like to suggest that we 
run PyPy.  We ourselves run PyPy in production.  It kinda works.


And yet, our test suite is still failing on PyPy after many years.

I'd like to suggest that if you have some time to write some code for 
Twisted this month, you fix a test failure or two on 
https://buildbot.twistedmatrix.com/builders/debian7-pypy2.5/builds/44/steps/trial/logs/problems 
https://buildbot.twistedmatrix.com/builders/debian7-pypy2.5/builds/44/steps/trial/logs/problems. 
 Some of these are subtle and weird, but there really aren't that many 
of them (14 failures, 16 errors in all).


Many of them appear to be just the fact that PyPy dicts retain order and 
seem like a great way to get started with contributing to Twisted.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Using six for Python3 porting

[Hynek Schlawack]

On 24 Apr 2015, at 8:41, Adi Roiban wrote:


I feel that twisted.python.compat is slowly duplicating / reinventing
an important part of the six code.


I’m +1 on this too.  I was a bit hesitant in the past but duplicating 
compatibility code everywhere is becoming ridiculous and a hassle.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Deprecating subproject packaging

[Hynek Schlawack]

 I think this is a good idea and am willing to put the work in to implement 
 this. Does anyone object to this? Is there a use case for keeping the 
 subprojects packagable that I don't know about?

+10

1. The split is due to Twisted not fitting on actual diskettes in ancient times.
2. It adds a frustrating amount of complexity.

—h
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] [ANN] pyOpenSSL 0.15

[Hynek Schlawack]

Greetings fellow Pythoneers,

I'm happy to announce that pyOpenSSL 0.15 is now available.

pyOpenSSL is a set of Python bindings for OpenSSL.  It includes some 
low-level cryptography APIs but is primarily focused on providing an API 
for using the TLS protocol from Python.


Check out the PyPI page (https://pypi.python.org/pypi/pyOpenSSL) for 
downloads.


***

This is the last release under the stewardship of Jean-Paul Calderone 
and the maintainership is now taken over by the Python Cryptography 
Authority (PyCA) which has been developing the C-bindings for pyOpenSSL 
for a while (aka cryptography).


We’d like to thank him for his great work over the past years and hope 
to be able to keep moving the project into a direction that will make 
him only slightly sad.


***

The highlights of this release include:

- Support to ECDHE,
- NPN and ALPN support,
- …many bug fixes!

It’s worth pointing out that OpenSSL functions generally work on *byte 
strings* because they mirror OpenSSL APIs and OpenSSL is not 
Unicode-aware.  Passing Unicode strings tends to accidentally work due 
do implicit decodes on Python 2 but they emit a DeprecationWarning now.  
Please note that DeprecationWarnings are silenced by default on Python 
2.7.


See the ChangeLog at 
https://github.com/pyca/pyopenssl/blob/0.15/ChangeLog for more 
details!


On behalf of PyCA,
Hynek Schlawack

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [ANN] pyOpenSSL 0.15*.1*

[Hynek Schlawack]

Hello again,

since releasing software is so much fun, 0.15.1 is out on PyPI too.

It fixes a small regression that shouldn’t affect you in practice but 
breaks the Twisted test suite.


See https://github.com/pyca/pyopenssl/pull/225 for details.

Brown baggily yours,
—h

On 14 Apr 2015, at 12:54, Hynek Schlawack wrote:


Greetings fellow Pythoneers,

I'm happy to announce that pyOpenSSL 0.15 is now available.

pyOpenSSL is a set of Python bindings for OpenSSL.  It includes some 
low-level cryptography APIs but is primarily focused on providing an 
API for using the TLS protocol from Python.


Check out the PyPI page (https://pypi.python.org/pypi/pyOpenSSL) for 
downloads.


***

This is the last release under the stewardship of Jean-Paul Calderone 
and the maintainership is now taken over by the Python Cryptography 
Authority (PyCA) which has been developing the C-bindings for 
pyOpenSSL for a while (aka cryptography).


We’d like to thank him for his great work over the past years and 
hope to be able to keep moving the project into a direction that will 
make him only slightly sad.


***

The highlights of this release include:

- Support to ECDHE,
- NPN and ALPN support,
- …many bug fixes!

It’s worth pointing out that OpenSSL functions generally work on 
*byte strings* because they mirror OpenSSL APIs and OpenSSL is not 
Unicode-aware.  Passing Unicode strings tends to accidentally work due 
do implicit decodes on Python 2 but they emit a DeprecationWarning 
now.  Please note that DeprecationWarnings are silenced by default on 
Python 2.7.


See the ChangeLog at 
https://github.com/pyca/pyopenssl/blob/0.15/ChangeLog for more 
details!


On behalf of PyCA,
Hynek Schlawack
--
https://mail.python.org/mailman/listinfo/python-announce-list

Support the Python Software Foundation:
http://www.python.org/psf/donations/


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted dinner at PyCon tonight (4/11)?

[Hynek Schlawack]

That clashes with the PyLadies auction.  :-/

I'm not sure I can attend anyway but it's unfortunate nevertheless. 

Sent from my phone.

 Am 11.04.2015 um 14:58 schrieb Ashwini Oruganti ashf...@twistedmatrix.com:
 
 Everyone is invited! Let's meet near the registration desk at 6:30pm
 
 Pullman
 3424, Avenue du Parc
 Montreal, QC H2X 2H5
 Canada
 (514) 288-7779
 
 http://www.yelp.com/biz/pullman-montr%C3%A9al-3
 
 -Ashwini
 
 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] getting rid of semantic newlines

[Hynek Schlawack]

On 5 Mar 2015, at 2:14, Glyph Lefkowitz wrote:

I find the semantic newlines standard which we have been attempting 
to enforce for documentation a constant source of annoyance.


Ostensibly, the purpose of using semantic newlines is to reduce the 
size of diffs.  However, given that we have oceans of documentation 
_not_ using this style, we are unlikely to reap that benefit any time 
soon.  Also, unlike code, documentation rarely needs small spot fixes; 
a fix to a document should result in changes elsewhere within the 
sentence or paragraph.


In pursuit of this questionable benefit, we have to accept the 
following annoyances:


It's inconsistent with pretty much every other Sphinx project out 
there.  Python lays out an 80-column maximum for Sphinx documents, the 
same as code: 
https://docs.python.org/devguide/documenting.html#use-of-whitespace 
https://docs.python.org/devguide/documenting.html#use-of-whitespace 
and an inspection of pretty much every other Sphinx project out there 
shows this style is consistently followed.


We don’t follow PEP8 either and that’s a much bigger annoyance to 
me.


It's inconsistent with the coding standard and requires special 
explanation in the docs.  I was prompted to write this message by 
attempting to review https://twistedmatrix.com/trac/ticket/7786 
https://twistedmatrix.com/trac/ticket/7786.
It requires special editor configuration.  ReST mode in emacs, vim, 
and sublime text expect to wrap paragraphs at 80 characters and 
keeping the semantic newlines where they're supposed to be has no tool 
support and involves avoiding other bits of tool support around 
re-flowing.  It also looks bad in an editor, with a ragged right edge.
It looks bad in online code browsers; long sentences horizontally 
scroll or wrap.


I think this style might have made sense ago 10 years ago in HTML, but 
with present-day RST it seems to go strongly against the grain.


Would anyone else like to make our documentation style-guide more 
harmonious with existing standards?  Anyone opposed?


I’m opposed; in general I have the same opinion as Donald: bad tooling 
shouldn’t dictate standards.


However I come to a very different conclusion: 80 chars for prose *is* a 
very artificial standard set by shitty tooling.  If we wanted to be 
consequent, it should be “enter only after paragraphs”.  Semantic 
newlines at least help me to quickly scan the structure of a document, 
they indicate when a sentence is too long etc.  In other words: it’s a 
concession to bad tools but at least it’s a useful one.  80 chars/line 
is just terrible in every regard and resulting from soft line wraps 
being NP-hard or something.


I’m not gonna veto it or something, just wanted to point out the 
tooling-inconsistency in the arguments that are frequently brought up.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Specifying ciphers in ssl.optionsForClientTLS

[Hynek Schlawack]

On 17 Feb 2015, at 2:52, Glyph Lefkowitz wrote:


I need to loosen up the default cipher list to allow RC4 (some sites
our customers use like myaccounts.socalgas.com still use it).

I was going to pass the following dict into the
extraCertificateOptions argument of ssl.optionsForClientTLS, but was
curious if there as a better way:

{acceptableCiphers : IAcceptableCiphers object}



As the documentation for extraCertificateOptions says, if you need to 
use it it's a bug in the interface.  As such, please file it :-).  
This escape-hatch was presented specifically so we could discover 
which features of that interface were really necessary customizations 
and which were just unfortunate compromises with OpenSSL's API.


In this case, no, there's no other way to get acceptable ciphers in 
there, and this should probably just be added to optionsForClientTLS.


Yes.

Another reasonable fix might be to allow RC4, since I think the 
default cipher suites that we have selected might be more appropriate 
for servers than for clients; the major browsers will still negotiate 
RC4 so we might want a slightly more permissive list.  Hopefully 
someone more cryptographically enlightened than I am can opine as to 
whether this is a reasonable thing to do in 2015...


I’m still maintaining that watering down the defaults is an invitation 
to downgrade attacks that affect *everyone*.  And RC4 looks worse with 
every month passing.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Looking for anyone interested in maintaining treq.

[Hynek Schlawack]

On 6 Feb 2015, at 10:53, HawkOwl wrote:

 I’d be interested in this :)

It’s not like you haven’t been maintaining it for months. :-P

 I think this also might be a good time to move it to the twisted/ org 
 alongside Klein, maybe?

Yes, we should discuss this.  getPage should be deprecated and treq should be 
pulled in.  Twisted is not a place packages go to die like the stdlib (although 
the barrier should be lower still).

signature.asc
Description: OpenPGP digital signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted ssl server and client

[Hynek Schlawack]

On 10 Nov 2014, at 11:32, John Aherne wrote:


So I am now looking at CertificateOptions in more detail.

But I am stuck trying to figure out how to add my GoDaddy cert to 
trustRoot.


You don’t need trustRoot for servers.  You need to supply an 
extraCertChain which – as I’ve mentioned before – is easiest with 
the help of pem: https://github.com/hynek/pem#twisted


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted ssl server and client

[Hynek Schlawack]

John,

On 7 Nov 2014, at 14:14, John Aherne wrote:


Thanks for the reply.

In the end I took the examples in the docs and changed them to fit.

So I have ended up with something that seems to work.

But I wouldn't mind if someone can tell me if what I have done is 
miles
wrong or spot on or could do with improvement or you have missed the 
point

completely.


I’m a bit confused as for what you’re trying to achieve.  Let me 
give you general pointers and maybe you’ll ask more specific questions 
afterwards.


I understand you want to use TLS both from a server and a client.  
Servers and clients have very different duties when it comes to TLS (and 
if you want to hear/learn more about them, you may want to take some 
time and watch my PyCon talk about it: 
https://www.youtube.com/watch?v=SBQB_yS2K4M ).


The *server* needs to make sure that its certificate chain is 
trustworthy, for that you need to load the certificate and the chain 
file you got from GoDaddy.  FWIW, you may want to use 
https://warehouse.python.org/project/pem/ for that because it takes some 
tediousness from it.


You should use some third-party application to verify you really got 
that right (don’t use the openssl CLI program, it’s confusing).


The *client* needs to verify the aforementioned certificate chain for 
its trustworthiness and whether it’s valid for the hostname you wanted 
to connect to.  In order to verify the trustworthiness, it requires a 
list of CAs it trusts.  One of them has to sign your final certificate 
in your chain file.


As glyph mentioned, loading CAs is a bit finicky and I have no 
experience on Windows unfortunately.  There is 
https://warehouse.python.org/project/wincertstore/ to extract them from 
the Windows store but I have no idea whether the output is useful with 
pyOpenSSL/Twisted.  A useful fallback is to use the bundle you get from 
https://warehouse.python.org/project/certifi/ .  If you’d like to help 
us to make this more friendly for Windows users we’d (and they!) would 
be eternally indebted. :)


It’s also worth noting, that you’re using the obsolete 
`DefaultOpenSSLContextFactory`, please use 
`twisted.internet.ssl.CertificateOptions` instead.  Is it possible, that 
you’re reading an older version of the TLS docs? Make sure to use 
http://twistedmatrix.com/documents/current/core/howto/ssl.html and also 
run Twisted 14.0.2 if you’re serious about using TLS.


Let us know if there’s something else unclear.

—h

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] 'module' object has no attribute 'optionsForClientTLS'

[Hynek Schlawack]

 I am trying to execute the check_server_certificate.py and I am getting this 
 error… can you guys help me figure out whats going on? its the exact sample 
 code on the website running on Ubuntu 14.04.

Trusty is shipping Twisted 13.2 which doesn't have vital TLS improvements that 
arrived with 14.0 and that are used in said example.

I strongly recommend against using Twisted older than 14.0 for code involving 
TLS.  Please use the version from PyPI (14.0.2).
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] naming question before we end up committed...

[Hynek Schlawack]

 Does anyone have an opinion on renaming it to twisted.logger?  Or have a 
 better suggestion?

+1 on dropping `python` here.  That's a weird namespace anyway something as 
central as logging shouldn't be there.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] 14.1 regressions (was: Python 3.3 buildslaves)

[Hynek Schlawack]

On 15 Sep 2014, at 8:16, Glyph wrote:

I noticed that https://twistedmatrix.com/trac/ticket/7355 is a 
blocker
for the release of Twisted 14.1 and #7355 is blocked Python 3.3 
buildbot

configuration/availability issues.


Thank you for (specifically) bringing this up and (generally) getting 
the 14.1 blockers cleared out. I'm beginning to notice a concerted 
effort :-).


Speaking of which:  I would greatly appreciate if someone with intimate 
knowledge of the test plumbing code could help me to figure out


https://twistedmatrix.com/trac/ticket/7370
and
https://twistedmatrix.com/trac/ticket/7429

***

In any case, as soon as the buildbot’s Python installations are 
updated to more recent versions of cryptography/pyOpenSSL our buildbots 
will turn red immediately.


Thanks,
Hynek

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] workflow

[Hynek Schlawack]

On 12 Sep 2014, at 8:20, Daniel Sank wrote:


I would greatly benefit from some direction regarding workflow. In
particular, how do you guys deal with having an installed version of
twisted, and a version (versions?) with which to tinker?

Currently, I have twisted installed in the normal end user way, and I 
have
a clone of my fork of the git repo. I mess around in the git repo 
clone,
but I don't know how to e.g. run the tests or run applications using 
the

git clone.

glyph explained this to me on irc a while back. Are the instructions 
from

that recorded somehwere?

Why am I asking? Because I want to do reviews and I'm not doing it 
because

I don't know how to set up a sane environment.


Like any other project: I have a virtualenv where I install Twisted’s 
git checkout with “pip -e .” plus all the dependencies I need.


I also have a tox.ini for common permutations of deps and Python 3, but 
unfortunately the tests tend to fail with weird heisenerrors at least on 
OS X so I can’t rely on that.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] running twisted with supervisord -- logging question

[Hynek Schlawack]

Jonathan,

On 24 Aug 2014, at 1:46, Jonathan Vanasco wrote:


Thanks for the insight.  I was stupid and didn't include my code.

I don't actually want supervisor to handle the logging.  I wanted to 
have this app's logs in /var/log/myapp-twisted/twisted.log


I just can't figure out how to make this happen.  i've been playing 
with different permutations of twistd commands, redirect_stderr and 
stdout_logfile, and haven't found the right balance


I would strongly urge you to do what Christopher told you, and not just 
for Twisted but in general.


Logging everything to stdout/stderr and then using some proper system 
tool to catch and process that instead of the finicky stdlib (or 
Twisted’s for that matter) logging will save you a lot of headaches 
and gain some love if you work with ops people.  Another alternative is 
using syslog which is directly supported by twistd.


—h

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Is there pb documentation somewhere?

[Hynek Schlawack]

On 8 Aug 2014, at 23:59, Glyph Lefkowitz wrote:


I've participated in this discussion several times:

Hypothetical Amalgam of Median Interlocutors Speaking Here: I'm using 
Tulip because I really like its style of coroutines.
Glyph: That's interesting. Did you know that Twisted has an 
equivalent style of coroutines, called inlineCallbacks, that's been 
around for years?
HAMISH: I saw that, and I asked about that a while ago and I heard it 
was bad.  It haven't heard that Tulip has the same problems, though.
Glyph: Really? What problems does inlineCallbacks have that Tulip's 
coroutines don't?
HAMISH: When I asked about it everybody told me I have to use 
Deferreds instead, but Deferreds are really confusing and they make 
your code look all gross, so I didn't want to do that.  With Tulip I 
don't have to!

Glyph: facepalm


That btw is something I’m trying to fight on IRC whenever I can for 
months now.  @inlineCallbacks may be worse than pure Deferreds in some 
ways, but they are amazing to get people to give Twisted a chance and 
start appreciating it (most people still have no clue what Twisted 
actually can do for them; hence the “who needs Twisted when we have 
tulip!?” questions).  And FWIW I have a mid-sized Twisted application 
running on top of @inlineCallbacks for years now and it works just fine.


People finally stopped knee-jerking at async/event-based programming and 
we’re keeping them out by being perfectionist smart-asses.  Next time 
someone asks about them, keep your “ugh inlineCallbacks” to 
yourself; a future contributor may come out of it.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] bringing LDAP back

[Hynek Schlawack]

On 7 Jul 2014, at 12:23, Bret Curtis wrote:


It is my recommendation that after this is reviewed and hopfully
commited that we make a branch and tag the release as 0.54.0. The
reason for the large jump is that antong's last semi-offical release
was 0.53 on PyPI. At this point we (myself and anyone else that wants
to help) should reach out to downstream projects (PyPI, Debian, and
etc.) to make them aware that Ldaptor development is again active.


Since ldaptor is a Twisted project now, may I suggest you copy its 
time-based version numbers? 14.0 has more meaning to itself than 0.54.0.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] bringing LDAP back

[Hynek Schlawack]

On 1 Jul 2014, at 12:14, Bret Curtis wrote:


With that said, lets get this ball rolling.

Firstly, we'll need a repo to get started with. My company (Amplidata)
has it's own fork, but I think it is best we start with a clone, not a
direct fork as github would have us do, of tv42s repo.


Agreed.


We can either:
1) Move (donate) tv42's repo to Twisted, this means that all links to
tv42/Ldaptor would automatically be forwarded to Twisted/Ldaptor.
2) Twisted creates it's own Ldaptor repo, I or someone else clones
this and then merge TV42's repo in, commit/push and file a merge
request with Twisted/Ldaptor.


Depends entirely on Tommi (cc’ed), I don’t care which route we take.


We're, of course, open to other suggestions, but those two above seem
the best options. From there, we can start dealing with other issues
such as:


0) Do the fixes to setup.py that everyone has in their private repos and 
put it on PyPI.



A) What to do with the UI part of Ldaptor. Who, if anyone, still uses
it? Do we trim it out or just mark it as deprecated since it relies on
old versions of twisted and nevow.


I wish we could just rip it out.  If there’s really a significant 
amount of people that use this, they build an ldaptor-ui package.



B) Pull in downstream patches from Redhat, SuSE and Debian.
C) Replace remaining bits of non-MIT code.
D) Get back to tv42's Todo list. :)


E) Start writing documentation. :(  *Something*. Currently there is only 
some slides and examples and it’s up to the user to read the source 
code and figure out what ldaptor actually can do (which is kind of *a 
lot*).


Cheers, I’m very glad we have some movement here,
—h

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Status of trac upgrade

[Hynek Schlawack]

On 23 May 2014, at 15:11, exar...@twistedmatrix.com wrote:

I was just wondering what the current status of the effort to upgrade 
trac on twistedmatrix.com is.


I could hear the crickets all the way down to Madagascar.

So what *is* the status?  The current state is really hardly bearable; 
the spam is taking completely over. :(  Wasn’t there a successful dry 
run at the PyCon sprints?


Cheers
Hynek

***

Also JFTR and related to our old plans to utilize GitHub somehow: it 
seems like Phabricator would be much rather worth our time as it allows 
for a complete review workflow: 
http://cramer.io/2014/05/03/on-pull-requests/


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[Hynek Schlawack]

On 7 May 2014, at 16:07, HawkOwl wrote:

 2 - Some work, medium risk - Release 14.0.0pre5 as 14.0 final,

I’m +1 on this one.  The pre5 has been widely tested and the only real issues 
are some embarrassing but inconsequential typos.  Let’s get this out before we 
introduce *real* problems.

 and I (or another RM if I’m no longer trusted ;) )

Nonsense, you’re doing great work.

 initiate the 14.1 release immediately.

You can try to do a 14.0.1 for all I care but those space errors IMHO don’t 
really warrant a point release (except that it would give us some practice for 
times when we need it as glyph pointed out on IRC).

signature.asc
Description: OpenPGP digital signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 14.0.0pre5 Announcement

[Hynek Schlawack]

On 1 May 2014, at 13:28, Glyph wrote:

I've upgraded https://glyph.im/ (and therefore 
https://glyph.twistedmatrix.com/ and https://pip2014.com/ and 
https://tm.tl/ and a number of other sites that nobody cares about) 
to the prerelease: https://asciinema.org/a/9216.


Smooth sailing so far, except for this one peculiarity; it crashes 
ssltest now:


https://www.ssllabs.com/ssltest/analyze.html?d=tm.tl

This might have nothing to do with the prerelease (for unrelated 
reasons I had to perform some other upgrades before I got around to 
it).


Also it looks like a bug on ssllabs' side of things, not a problem 
with Twisted.  But if someone slightly more experienced with TLS 
wanted to look at the traffic from that server it might be 
interesting.


When I connect to the hosts you mention using openssl (don’t forget to 
set -servername if you play along) I only get TLSv1.  Is it possible 
that there’s some custom TLS code laying around?


—h

P.S. The cert chain is apparently completely wrong: 
http://glui.me/?i=ek3zvx7v2wrlsgm/2014-05-01_at_13.55.png/  Apparently 
you send out an anchor but missing an intermediate certificate?


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 14.0.0pre5 Announcement

[Hynek Schlawack]

On 1 May 2014, at 21:28, Glyph Lefkowitz wrote:

When I connect to the hosts you mention using openssl (don’t forget 
to set -servername if you play along) I only get TLSv1.  Is it 
possible that there’s some custom TLS code laying around?


As far as I can see, only https://github.com/glyph/txsni.  It 
constructs the CertificateOptions in 
https://github.com/glyph/txsni/blob/master/txsni/only_noticed_pypi_pem_after_i_wrote_this.py 
(whose name suggests a change I need to make to this library).  Am I 
forgetting some cool new options to CertificateOptions?


If you want DHE, you need to load DH parameters: 
http://twisted.readthedocs.org/en/latest/core/howto/ssl.html#tls-protocol-options 
too.


Why your server only accepts TLSv1 is beyond me off the cuff.

The machine is an Ubuntu 14.04 machine with 
libssl1.0.0:libssl1.0.1f-ubuntu-don't-have-a-heart-attack-it's-actually-g 
(I seriously wish they wouldn't do that with security patches).


Well, that’s what distributions do. *shrug*  They don’t update your 
software so nothing breaks; they just fix the security issues (thus 
it’s not necessarily g, Ubuntu’s fix *can* be very different from 
what OpenSSL did.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 14.0.0pre5 Announcement

[Hynek Schlawack]

On 29 Apr 2014, at 12:16, HawkOwl wrote:

 If anyone else has some applications they run, please try out pre5!

Look at all the ECDHEHEHE: “gnutls-cli -s --crlf  imap.variomedia.de -p 4190”  
(and DHEHEHEHE too if you add “--priority='PFS:!ECDHE-RSA'”).

Tests are passing everywhere too.

 On 29 Apr 2014, at 2:33, David Reid dr...@dreid.org wrote:

 Sorry for the delay, I've tested that treq works with pre5 in the following 
 scenarios.

 Python 2.7, no pyopenssl
 Python 2.7, pyopenssl==0.14

 PyPy 2.2.1, no pyopenssl
 PyPy 2.2.1, pyopenssl==0.14

 This is in addition to the testing that treq regularly receives on Twisted 
 Trunk, which has a build currently running:

 https://travis-ci.org/dreid/treq/builds/23955777

 -David


 On Sat, Apr 26, 2014 at 1:35 AM, HawkOwl hawk...@atleastfornow.net wrote:
 Hi everyone, just a few changed bytes to fix some news file entries.

 Tarballs for this prerelease can be found at 
 http://twistedmatrix.com/Releases/pre/14.0.0pre5, with the changelog at 
 http://twistedmatrix.com/Releases/pre/14.0.0pre5/NEWS.txt.

 Changes from the previous prerelease include:

  - Newsfile fixes.

 For more information and a full list of changes, check the NEWS.txt file.

 Please download the tarballs and test them with your applications, so we can 
 make sure we’re all ready for release!

 -hawkowl

 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


signature.asc
Description: OpenPGP digital signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 14.0.0pre5 Announcement

[Hynek Schlawack]

That's a Hawkowlism. :)

Sent from my phone.

 Am 29.04.2014 um 19:43 schrieb Glyph gl...@twistedmatrix.com:
 
 
 On Apr 29, 2014, at 3:44 AM, Hynek Schlawack h...@ox.cx wrote:
 
 If anyone else has some applications they run, please try out pre5!
 
 Look at all the ECDHEHEHE: “gnutls-cli -s --crlf  imap.variomedia.de -p 
 4190”  (and DHEHEHEHE too if you add “--priority='PFS:!ECDHE-RSA'”).
 
 Okay I can't even tell when TLS things are a joke any more.
 
 Is 'ECDHEHEHE' a real cipher suite or are you just laughing? ;-)
 
 -g
 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Making twisted web client requests on running reactor

[Hynek Schlawack]

Hello Ameya,

On 23 Feb 2014, at 3:17, Ameya Lokare wrote:

I'm writing a client library that makes (potentially long running) 
HTTP
requests. Since the library will be used within non-Twisted code, I 
was

thinking of running the reactor in a separate thread.


Have you seen Crochet? That might be just what you’re looking for: 
https://pypi.python.org/pypi/crochet/


Cheers
Hynek

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] starting point to contribute

[Hynek Schlawack]

Hi Kasun,

On 18 Jan 2014, at 4:57, Kasun Thennakoon wrote:

Thanks for your attention.  I'm more interested in GPS protocols and 
web

protocols.


If you’re into GPS, you may enjoy helping 
https://twistedmatrix.com/trac/ticket/3926 getting merged at last!


Cheers,
—h

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Filing Bugs

[Hynek Schlawack]

On 5 Jan 2014, at 19:52, Daniel Sank wrote:


I have a windows computer and an Ubuntu computer that I use as a file
server. I'm reading about IRC now. How are we supposed to find each 
other?


You could try Freenode’s web chat: https://webchat.freenode.net so you 
don’t have to extra install software.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] txdlo - a Twisted DeferredListObserver

[Hynek Schlawack]

On 30 Dec 2013, at 6:24, Terry Jones wrote:

 I'll improve the examples (and probably add tests for them) at some point.
 Meanwhile, the DeferredListObserver class (in txdlo.py), tests for it, and
 various examples can be had from https://github.com/terrycojones/txdlo

That’s great! Any chance of adding a license (*cough* MIT *cough*) and putting 
the whole thing on PyPI?

signature.asc
Description: OpenPGP digital signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] txdlo - a Twisted DeferredListObserver

[Hynek Schlawack]

Hey Terry,

On 30 Dec 2013, at 21:20, Terry Jones wrote:


Any chance of adding a license (*cough* MIT *cough*)
Is Apache ok for you?  If not, mail me off list and I can do MIT for 
you.


Apache is just fine for me – I use it for structlog too. I was 
suggesting MIT because that’s Twisted’s license.



and putting the whole thing on PyPI?


See https://pypi.python.org/pypi/txdlo

I added a proper README, more tests, the license, setup.py etc. See
https://github.com/terrycojones/txdlo


Great, now I can play with it!

—h

P.S. if I can pester you any further: I’d appreciate a wheel and you 
can have a more useful PyPI entry by including the README – see 
self-promotion 
https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/ 
/self-promotion


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Endpoints and WebSocket / Autobahn

[Hynek Schlawack]

On 31 Dec 2013, at 0:22, Tobias Oberstein wrote:


Is there an example of doing this with 'twistd' anywhere?


I was hoping you or someone could tell me how to do one;)

Thing is: I was looking for --endpoint ... argument with twistd, but 
there isn't.


How is the full endpoint vision supposed to work out? I mean, being 
able to:


twistd conch --endpoint autobahn:tcp\:80:url=ws\://myhost.com

to run a conch server that can be accessed running SSH over WebSocket 
over TCP.


FWIW, the argument you’re looking for is usually called “--port”.  
The most obvious example is the web plugin.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] mind introduced strangely in pb howto

[Hynek Schlawack]

Am 24.10.2013 um 09:02 schrieb Tobias Oberstein tobias.oberst...@tavendo.de:

 I just tried to register so I could do that. When I clicked on the register 
 button
 after filling out the username/password fields my browser (firefox) brought
 up a notice that the security certificate is invalid because of unavailable
 issuance chain information. Knowing absolutely nothing about internet
 security issues I thought I should mention this and ask if this is expected
 behavior.
 
 I wouldn't call that expected behavior, since
 
 a) the certificate used on twistedmatrix.com contains (as it should) 
 intermediate CA certs also (see attachments)

I’m not sure what you mean with “contains”? It certainly *relies* on one but 
unfortunately doesn’t send it along (yet):

$ openssl s_client -host www.twistedmatrix.com -port 443



 
CONNECTED(0003)
depth=0 
/description=S7lbCt7N2R4t9o8J/C=US/CN=www.twistedmatrix.com/emailAddress=postmas...@twistedmatrix.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 
/description=S7lbCt7N2R4t9o8J/C=US/CN=www.twistedmatrix.com/emailAddress=postmas...@twistedmatrix.com
verify error:num=27:certificate not trusted
verify return:1
depth=0 
/description=S7lbCt7N2R4t9o8J/C=US/CN=www.twistedmatrix.com/emailAddress=postmas...@twistedmatrix.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 
s:/description=S7lbCt7N2R4t9o8J/C=US/CN=www.twistedmatrix.com/emailAddress=postmas...@twistedmatrix.com
   i:/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom 
Class 1 Primary Intermediate Server CA
---


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] mind introduced strangely in pb howto

[Hynek Schlawack]

Am 24.10.2013 um 08:08 schrieb Daniel Sank sank.dan...@gmail.com:

 When I clicked on the
 register button after filling out the username/password fields my
 browser (firefox) brought up a notice that the security certificate is
 invalid because of unavailable issuance chain information. Knowing
 absolutely nothing about internet security issues I thought I should
 mention this and ask if this is expected behavior.

This will be fixed as soon as the now-in-prerelease Twisted 13.2 has been 
deployed to Twisted’s homepage (i.e. hopefully soon). Earlier versions don’t 
allow the specification of chain certificates unfortunately so it’s up to the 
browsers to fetch them – or not.


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] mind introduced strangely in pb howto

[Hynek Schlawack]

Am 24.10.2013 um 09:48 schrieb Tobias Oberstein tobias.oberst...@tavendo.de:

 FWIW, you can manually concatenate certs .. this is what we do (also for 
 StartSSL):
 
$ cat myserver_plain_cert.crt  myserver.crt
$ cat ../sub.class1.server.sha2.ca.pem  myserver.crt
$ cat ../ca.pem  myserver.crt
 
 A concatenated cert like above works today without the new code that is 
 upcoming in Twisted. Which is cool also.

That is completely new to me. Are you sure you’re not mixing up Twisted’s 
behavior with nginx?

If what you say is true, there would have never been the need for #2061 and the 
monkey patching everyone was doing before it landed. Can you point me at a 
server where you have deployed TLS like that please?


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] mind introduced strangely in pb howto

[Hynek Schlawack]

Am 24.10.2013 um 12:44 schrieb Tobias Oberstein tobias.oberst...@tavendo.de:

 Your server definitely sends three certificates - that's 
 surprising/confusing.
 
 Could you double-check how you've achieved that? If you google for chain
 certs  Twisted you'll find all kinds of monkey patches to achieve that; and
 when I run twistd -n web with a pem that has multiple certificates I still 
 get
 sent only one from the server. I feel like I'm missing something.
 
 Ok, sorry, I forgot totally about it .. but this is what we do:
 
 https://github.com/crossbario/crossbar/blob/master/crossbar/crossbar/tlsctx.py#L73
 
 It indeed relies on use_certificate_chain_file.
 
 Sorry. My fault: it needs patching.

Phew, you really got me sweating there. :)

Since you’re not using string representations there, you can move to 
CertificateOptions which has chain files sind 13.1 already (the string support 
slipped into 13.2 because I don’t know how to Python and it got reverted a few 
days before the release of 13.1 because of a Python 3 regression).


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Sprint Report (2013-09-23)

[Hynek Schlawack]

Am 30.09.2013 um 20:42 schrieb Glyph gl...@twistedmatrix.com:

 I have that on my todo-list since the beginning of time but never got around 
 to it.
 
 Do you have a link to a Nevow bug in some tracker (launchpad, perhaps?) that 
 is tracking this?

Yes, I believe it’s https://bugs.launchpad.net/nevow/+bug/1091055


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Sprint Report (2013-09-23)

[Hynek Schlawack]

 * Dev Requirements
 It would be nice to have a dev-requirements.txt file so that they
 could easily install the necessary development tools. pydoctor,
 coverage, nevow, zope.interface, twistedchecker, etc
 See:
 * https://github.com/hynek/structlog/blob/master/dev-requirements.txt
 
 What tools actually make use of this file?  

pip install -r dev-requirements.txt

 Why is it desirable to keep this information there, instead of the 
 'requirements' key in setup.py?

There is no ‘requirements’ key – and ‘requires’ are runtime dependencies.  You 
don’t want to put your development tools there.

   (Is there a 'develop_requires' key?)

No: 
http://pythonhosted.org/distribute/setuptools.html#new-and-changed-setup-keywords

There is tests_require which is also something different from development tools 
and get used when you run setup.py test which we AFAIK don’t.


signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [Twisted] #6663: Allow CertificateOptions to set acceptable SSL ciphers

[Hynek Schlawack]

  1. That there is a consent on high quality ciphers: for example right
 now there are roughly two fractions who agree what is the lesser evil: RC4
 or AES-CBC.
 
 No, it is now clear that RC4 is the greater evil. The browsers have
 deployed defenses against the BEAST attack on CBC (the defense is 1/n-1
 record splitting), and BEAST is an active attack which can only be used
 in some cases and which tends to leave evidence of the attempt. On the
 other hand, RC4 is apparently vulnerable to passive attacks, which are
 more serious.
 
 (If I'm wrong and there actually *is* a faction who still prefers RC4
 despite the recent results against it, I'd like to read about it!)

I’m not going to argue ciphers with you because you’re obviously right and I 
already wrote elsewhere that I’m going to full defer to your judgement here.

To explain where the above came from and eg. Qualys is still somewhat for RC4 
as a fallback cipher: to the best of my knowledge[1], Apple’s desktop Safari 
browser ''still'' hasn’t activated record splitting in its latest version and 
is thus still vulnerable to BEAST (and doesn’t support TLS1).  But that’s 
probably a corner case enough to ignore in the defaults and will hopefully 
resolve itself in Mavericks.

[1]: Mostly from 
https://community.qualys.com/blogs/securitylabs/2013/03/19/rc4-in-tls-is-broken-now-what
 and I’m not aware of any changes.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [Twisted] #6663: Allow CertificateOptions to set acceptable SSL ciphers

[Hynek Schlawack]

please disregard this mail I mixed up the behavior of roundup and trac.

feel free to comment on ticket #6663 though.

Am 16.08.2013 um 08:19 schrieb Hynek Schlawack h...@ox.cx:

 1. That there is a consent on high quality ciphers: for example right
 now there are roughly two fractions who agree what is the lesser evil: RC4
 or AES-CBC.
 
 No, it is now clear that RC4 is the greater evil. The browsers have
 deployed defenses against the BEAST attack on CBC (the defense is 1/n-1
 record splitting), and BEAST is an active attack which can only be used
 in some cases and which tends to leave evidence of the attempt. On the
 other hand, RC4 is apparently vulnerable to passive attacks, which are
 more serious.
 
 (If I'm wrong and there actually *is* a faction who still prefers RC4
 despite the recent results against it, I'd like to read about it!)
 
 I’m not going to argue ciphers with you because you’re obviously right and I 
 already wrote elsewhere that I’m going to full defer to your judgement here.
 
 To explain where the above came from and eg. Qualys is still somewhat for RC4 
 as a fallback cipher: to the best of my knowledge[1], Apple’s desktop Safari 
 browser ''still'' hasn’t activated record splitting in its latest version and 
 is thus still vulnerable to BEAST (and doesn’t support TLS1).  But that’s 
 probably a corner case enough to ignore in the defaults and will hopefully 
 resolve itself in Mavericks.
 
 [1]: Mostly from 
 https://community.qualys.com/blogs/securitylabs/2013/03/19/rc4-in-tls-is-broken-now-what
  and I’m not aware of any changes.
 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] SQL ORM for Twisted PostgreSQL?

[Hynek Schlawack]

Am 25.07.2013 um 00:43 schrieb Glyph gl...@twistedmatrix.com:

 If you just want to see broader testing first, a better solution would be for 
 Hynek to just contribute to the Calendar Server project directly so that 
 there are effectively 2 parties involved rather than 3; for starters, we 
 could have a setup.py in the calendarserver.org repository that just submits 
 twext.enterprise.* to PyPI as an independent source distribution, while 
 remaining part of the same project.  It can *appear* to be a separate project 
 as long as it's developed in the same place. :-)

I don’t really care *how* it gets to PyPI – all I was suggesting to get it 
there *somehow* so people can get excited about it.___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] SQL ORM for Twisted PostgreSQL?

[Hynek Schlawack]

Am 24.07.2013 um 20:05 schrieb exar...@twistedmatrix.com:

 It seems like this would get the ball rolling more quickly and I don't see 
 the downside.  If anything, having it as an independent project is more 
 likely to get more people interested in it more quickly and so increase the 
 pool of people who might be interested in addressing whatever concerns need 
 to be addressed for its inclusion in Twisted.

That’s exactly my line of thought here.  adbapi2 has been pointed out several 
times now on this list but I doubt people really use it since it’s rather 
cumbersome to get.

I’m not fighting/delaying its inclusion in Twisted, I just don’t see that 
happen effectively anytime soon even if the integration work started 
immediately.  I would just add a smaller step in between that may provide 
additional insights about potential usage + add some momentum.

Cheers,
—h
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] SQL ORM for Twisted PostgreSQL?

[Hynek Schlawack]

Am 22.07.2013 um 23:37 schrieb Glyph gl...@twistedmatrix.com:

 How would you feel about packaging it up on PyPI so people can try it out 
 effortlessly? What do Apple’s licenses say about that? Yes, I’m volunteering.
  
 It seems it's released under the ASL2. I don't know if Apple prevents *its* 
 employees from doing anything in particular, but it seems like third party 
 contributors are free to do with it as they please (within the limits of the 
 license, of course). 
 
 There are actually bits of this that I've been meaning to contribute back for 
 a long time.  If somebody would like to help me out with it then maybe I can 
 eke out a little bit of time to do it - I've just been pretty busy :-).

What exactly do you mean? Sounds like you’d like it to got straight into 
Twisted? Wouldn’t it make more sense to release it separately first? You know, 
kind of http://www.codinghorror.com/blog/2013/07/rule-of-three.html 

We could move it into the twisted namespace though.___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] logging

[Hynek Schlawack]

Am 15.06.2013 um 08:18 schrieb Christopher Armstrong ra...@twistedmatrix.com:

 As far as actual *proposals* go, I have these ones, that are all independent:
 
 1. include all keyword arguments in log output without requiring
 specifying the formatting in the string
 2. make it really easy to specify the system
 3. stop affecting the system of application code based on the
 framework code that's running the application code (i.e., don't use
 callWithContext to specify the system any more)
 
 Of these, I think #2 and #3 have the most benefit; I can do #1 with my
 own logging statements just fine, and while IMO it'd be nice if the
 whole world adopted a nice identifier-based system, the lion's share
 of the benefit comes from my use of it consistently in the
 application's codebase.
 
 This conversation has gotten pretty sprawling; time to reel it in with
 some code.
 
 What do you think of this for an API that meets in the middle?
 
 https://gist.github.com/radeex/5787124
 
 This example implementation only concerns itself with the points under
 debate right now; obviously it's completely unusable in general. But
 anyway:
 
 1. it supports English logs
 2. it doesn't require you to specify a formatting if you want to just
 log a bunch of data
 3. it makes it easy to specify a system (both manually and based on
 the module your code is in)

I’ve held back from this discussion so far because it seemed to me that I 
always missed some part of the discussion to fully understand what you’re all 
talking about. I would like to comment on this concrete proposal though before 
I hold my peace forever. (NB I’m not replying just to Christopher but try to 
address everything I saw on the thread so far – I like most of his proposal.)

I find that there’s some kind of false dichotomy brought up in this discussion 
and API and output are somewhat muddled together a bit (maybe I’m just 
misunderstanding though – that’s why I didn’t comment until now).

I personally like my logs 100% structured (except for Exceptions) and still be 
able to “comment” on events in plain English if I need to.

And I don’t see why comments/events should be special case on output (square 
brackets in this example). If you have an event called user_error, you can 
always add a key called error for another “symbol” or just an error_msg if you 
insist on English. When looking for a certain type of user_error, you simply 
write an AND clause in your logging software (Kibana or whatever). It’s pretty 
easy to keep *that* consistent across applications.

For example, *my* log would look like this:

event=user_error peer=127.0.0.1 error=pebkac

If the programmer in question hadn’t enough logging pain in their life to see 
that’s reasonable, they can always do:

event=user_error peer=127.0.0.1 error_msg='Problem exists between keyboard and 
chair.'

Still perfectly parseable, perfectly readable. And with {!r} easy to achieve. A 
nice API I would like to have be:

log('user_error', peer=self.transport.getPeer().host, error_msg='Problem exists 
between keyboard and chair.') – and log figures out itself if it needs to 
quote. I could also live with them all quoted, i.e.:

event='user_error' peer='127.0.0.1' error='pebkac'

to have less special cases.


I hope that makes some sense, the point I’m trying to make that events don’t 
need to be distinct by themselves. If you enforce that, you’re forcing 
structure on them which you could spread over multiple fields that are *much* 
more pleasant to parse.

Regards,
Hynek

P.S. For convenience, I usually write a log method in Twisted protocols that 
prepends the messages with state data, peer IP etc, but that JFTR.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Web documentation out of date

[Hynek Schlawack]

 Twisted reviewers: review tickets!  Tom, the recipient of the sponsored 
 fellowship, is busy with fixing our infrastructure, so we really need the 
 community to pitch in.
 
 If you're interested in reviewing tickets, please let me know.

I spoke about that to jp already: collecting some *practical* material on 
proper code review in the wiki would be really beneficiary. I've been pointed 
to http://mumak.net/stuff/your-code-sucks.html which is a nice start but 
doesn't one really get started. Maybe a checklist to hold to in the beginning? 
Pointing out common things to look for…___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Current callback best practices

[Hynek Schlawack]

Am 02.04.2013 um 16:32 schrieb Glyph gl...@twistedmatrix.com:

 My question can be simplified to: Closures yes or no?
 As appropriate.

I had that coming. :)

[…]

Thanks for your verbose explanations!

 If “yes, closures”: Still using cb/eb prefixes? I don’t see them very often 
 in recent examples. What about addBoth handlers?
 If you're using a closure, use a descriptive name.  The 'cb/eb' prefix 
 notation is not particularly illustrative; I often use an english word like 
 'when'.  whenDisconnected, whenResponseReceived, etc.

So you’re cool with calling callbacks after the event that triggered? That 
would’ve been my second question but I didn’t want it to divert the discussion. 
I kind of like `onData` et al too but the Twisted examples rather shun it.

Thanks,
Hynek

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Current callback best practices

[Hynek Schlawack]

Hi,

I’ve asked this one already at the Twisted dinner (thanks again for it, it was 
probably my PyCon highlight) and got various answers from people sitting around 
me. I’m still indecisive so I’d like to gather even more opinions[1]. I already 
tried to ask on IRC but my time zone is just not very #twisted friendly so I 
got zero replies.

My question can be simplified to: Closures yes or no?

They seem *really* handy since they allow to have some data present without 
handing it through all the time. Eg in my cred checker, I can refer to the user 
name from the closure instead of passing it around all the time – making the 
code much cleaner. Also most current examples and 
callbacks-are-hard-FUD-rebuttals seem to use them.

OTOH, private methods like `_cbPrintResult` are nicer to test individually.

If “yes, closures”: Still using cb/eb prefixes? I don’t see them very often in 
recent examples. What about addBoth handlers?

Bonus points for the preferred style for the Twisted code base.

Thanks in advance  thank you for your time,
Hynek

[1]: I’d be glad if people from the dinner replied again too so I/we have it in 
written form, my memory tends to be rather sloppy.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted dinner pictures

[Hynek Schlawack]

Great, now make it coherent and quotable please. :)

And the others: please help out too! I won’t name people I know about having 
pictures because they may not want me to share them with the world – but know 
that I’m sad. ;)

P.S. Does anyone have a picture of “Go Ashwini”?

Am 26.03.2013 um 16:11 schrieb Laurens Van Houtven _...@lvh.cc:

 I think that last year's GSoC student came in not knowing any Twisted and was 
 a speaker this year that picked up a release manager hat just to get me to 
 merge my 5 year old branch is a testament to:
 
 - open source is awesome
 - sprints are awesome
 - pycon is awesome (even if my phone insists on correcting it to toxin)
 - ashfall is awesome
 
 On Mar 25, 2013 7:08 PM, Hynek Schlawack h...@ox.cx wrote:
 Hi,
 
 As you may have noticed, I'm building a little page to show off that PyCon 
 wasn't about donglegate. http://thisispycon.com
 
 I would love to post about our little dinner and would be delighted if anyone 
 having pics AND quotes could forward them to me.
 
 Same goes for sprints etc, flood me!
 
 Cheers,
 
 Hynek
 
 Sent from my phone.
 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
 ___
 Twisted-Python mailing list
 Twisted-Python@twistedmatrix.com
 http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Twisted dinner pictures

[Hynek Schlawack]

Hi,

As you may have noticed, I'm building a little page to show off that PyCon 
wasn't about donglegate. http://thisispycon.com

I would love to post about our little dinner and would be delighted if anyone 
having pics AND quotes could forward them to me.

Same goes for sprints etc, flood me!

Cheers,

Hynek

Sent from my phone.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Must avatarId always be a string?

[Hynek Schlawack]

Am 14.01.2013 um 23:58 schrieb Glyph gl...@twistedmatrix.com:

 It seems like the shared caching reference would solve this problem
 as well?
 
 Yes, I think that's the right answer. It's certainly the right design
 in my case, and perhaps in the general case too.
 
 I think that this is the right design for now, given the constraints of the 
 current cred API; however, it's certainly possible we could think up an 
 extension to the cred API that would make this case easier to deal with.  It 
 also leaves open some not-quite-trivial questions like how do you know when 
 to expire the cache, which are usually easy to work out for a particular 
 application but do not generalize easily.

I was under the impression that the IRealm.requestAvatar() is *always* called 
*once* after ICredentialsChecker.requestAvatarId() so I simply delete the data 
from the cache once my avatar is created (and it works fine).

Am I presuming something I shouldn’t?___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Must avatarId always be a string?

[Hynek Schlawack]

Am 09.01.2013 um 14:04 schrieb Jan Urbański wulc...@wulczer.org:

 http://twistedmatrix.com/documents/current/core/howto/cred.html says
 that the avatarId parameter to IRealm.requestAvatar must be a string,
 not even a Unicode string. I'm using LDAP for authentication, and the
 checker retrieves the full LDAP entry for the user as a side-effect of
 authentication.
 I remember discussing this on IRC with someone

;)

 not long ago and he 
 pointed me to this thread:
 
 http://twistedmatrix.com/pipermail/twisted-python/2010-September/022826.html
 
 I have faced a similar problem myself and after reading the code I've 
 resolved to wilfully disregarding the documentation and passing tuples 
 around, accepting that if it breaks, I get to keep both pieces.

Since my final approach wasn’t mentioned yet: I found it semantically weird to 
pass the full data in a variable called “avatarId”, so I went with a dict as a 
cache that both the checker and the realm get passed in on construction. Not 
pretty but in my eyes still prettier. :)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Making ConnectionPool not pool

[Hynek Schlawack]

 I have a really bad time with the combination of a low-volume service  and 
 adbapi.ConnectionPool, pyodbc, FreeTDS and Sybase.
 
 Basically my connections just time out and fail in weird, generic ways like:
 
 Error: ('01000', '[01000] [FreeTDS][SQL Server]Unexpected EOF from the server 
 (20017) (SQLEndTran)')
 
 (but in many others too, there is no real pattern)
 Have you tried enabling reconnects (cp_reconnect=True)? If select 1 doesn't 
 work with your database you may also have to pass in custom cp_good_sql.

Yep, and the connections *do* heal after dying. I don't have to restart twistd, 
it just takes a few failed requests to warm up again.

I tried to do a for loop like:

succ = False
for _ in range(self.dbpool.max + 1):
try:
yield self.dbpool.runOperation(self.dbpool.good_sql)
succ = True
except Exception:
pass

if not succ:
raise pyodbc.Error(b'Found no healthy connection.')

…for new connections, but that helped only a bit. I presume once it finds a 
working connection, it keeps using it. But I still had failure within the 
session later.

 In my desperation, I’m employing for-loops for the SQL queries now. :(
 
 Since there isn’t much traffic (yet) I would like to just make ConnectionPool 
 close the connections and re-open fresh ones, as soon as they are necessary.
 
 Is there some straight-forward way to do that? Or any better approach I’ve 
 overlooked?
 Don't use ConnectionPool at all. Just have a function that does the SQL 
 connect etc usually normal DB-API, and call it with 
 twisted.internet.threads.deferToThread:
 
  def dbTxn(x):
 conn = db.connect(...)
 cursor = conn.cursor()
 cursor.execute()
 result = cursor.fetchall()
 conn.close()
 return result
 
 deferredResult = deferToThread(dbtxn, argForX)

Sounds like a solid backup plan, thanks!

The only reason I’d prefer to stay with pooling is that I very much expect the 
traffic to rise and wouldn’t really want to change away and go back later. :(

Cheers,
Hynek___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Making ConnectionPool not pool

[Hynek Schlawack]

Am 27.11.2012 um 14:39 schrieb exar...@twistedmatrix.com:

 The only reason I’d prefer to stay with pooling is that I very much 
 expect the traffic to rise and wouldn’t really want to change away and 
 go back later. :(
 Then I'd strongly recommend taking a look at twext.enterprise:
 
 http://trac.calendarserver.org/browser/CalendarServer/trunk/twext/enterprise
 
 There's a good chance this will become part of Twisted as a replacement 
 for twisted.enterprise at some point.  So any experience you get with it 
 now should serve you well in the future.  Plus, if you find it also 
 can't deal with your situation, then it'd be nice if we knew that now. 
 :)

Gladly! Do I see it correctly, that the current canonical way to use it, is to 
svn co it?
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Ideas for a Twisted Training

[Hynek Schlawack]

Hi,

On Fri, Jun 17, 2011 at 12:05, Thomas Hervé the...@free.fr wrote:

 Also, if anyone else from the Twisted community is going to be there, let's 
 have a drink.
 I'll be around starting Tuesday, happy to meet you here!

Will there be a Twisted table at PyBeer/PyFiorentina? :)

Cheers,
-h, attending.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Anyone ever used successfully sqlanydb together with Twisted and adbapi?

[Hynek Schlawack]

Hi,

I've essentially written a whois daemon that serves via TCP as well as
via web, fetching its data from a sqlanywhere 12 server, using the
official sqlanydb driver.

Everything works fine except for occasional SIGABRTs (or inside of
gdb: SIGSEGVs inside of the sqlanywhere binary driver) in the first
query. If that one works, it keeps running.
 However it's impossible to run tests as at some point it always
crashes while doing queries because I recreate the pool for every
test.

Essentially I get a domain record from the database and fetch some
related data afterward. I tried to implement it using DeferredList and
using runInteraction and it always aborts before doing the second
query. I've already posted some code and gdb backtraces to
stackoverflow: 
http://stackoverflow.com/questions/5790435/python-twisted-sqlanydb-abort
maybe there's some helpful data I missed here. To stress it: The
lookup of the domain record usually works. It's the following defers
that crash my application.

My essential question is: Has someone already used Twisted and
SQLAnywhere together successfully? Is it me or is it sqlanywhere or
even Twisted? What are my options?

TIA,
-h

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python