[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

Thanks a lot Victor for your numerous reviews and your help on this!
Also thanks for all other who commented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Roundup Robot]

Roundup Robot added the comment:

New changeset 42e7334e0e4c by Victor Stinner in branch '3.4':
Issue #23630: Fix test_asyncio on Windows
https://hg.python.org/cpython/rev/42e7334e0e4c

New changeset 840942a3f6aa by Victor Stinner in branch '3.5':
Merge 3.4 (Issue #23630, fix test_asyncio)
https://hg.python.org/cpython/rev/840942a3f6aa

New changeset 1f979a573f8d by Victor Stinner in branch 'default':
Merge 3.5 (Issue #23630, fix test_asyncio)
https://hg.python.org/cpython/rev/1f979a573f8d

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

I pushed the change to Python 3.4, 3.5, 3.6 and asyncio at Github.

Thanks Yann for your nice enhancement of asyncio! It put your name in CPython 
"acks" and asyncio author list.

I modified the patch a little bit (3 lines), how the hosts variable was 
initialized. That's all :-)

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Roundup Robot]

Roundup Robot added the comment:

New changeset 682623e3e793 by Victor Stinner in branch '3.4':
Issue #23630, asyncio: host parameter of loop.create_server() can now be a
https://hg.python.org/cpython/rev/682623e3e793

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

Yet another version of the patch according to the last review.

--
Added file: http://bugs.python.org/file40366/multibind8.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

Good. I have less comments, it's getting better at each iteration ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

A new (and hopefully last?) version of the patch, thanks again for the review 
Victor!

--
Added file: http://bugs.python.org/file40356/multibind7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

Here is a new patch without any dependency on netifaces.
Thanks Victor for the great deal of help for all the mock() stuff!

--
Added file: http://bugs.python.org/file40342/multibind6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

Yann, I'll commit this as soon as 3.5.0 and 3.4.4 are out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

> Let's not extend the API with support for multiple ports. Nobody has asked 
> for it, and clearly the API design is not so simple.

Ok. Anyway, the user of the API call call create_server() multiple times, even 
with the current code. So it's fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

> - accept host and port arguments being sequences and then we bind to all 
> host:port combinations. Like if len(host) == N and len(port) == M, we bind to 
> N*M sockets.

I can't think of a real life example for this.  But if you need that, you can 
always implement that on top of function that accepts multiple (host,port) 
pairs.

> - accept host and port arguments being sequences and allow repetitions and 
> bind to host:port couples from zip(host, port).

I'm not sure I like this either.  This breaks one value `(h,p)` into two 
values, and passes them in separate collections to separated arguments.  Too 
complex.

My proposal is to allow the following:

   create_server(protofac, [(h1,p1),(h2,p2)])

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

Let's not extend the API with support for multiple ports. Nobody has asked for 
it, and clearly the API design is not so simple.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

"So I like the idea of supporting multiple ports. It's very cheap to support 
it."

Oh, I had an example in my head: listening on tcp/80 for HTTP and tcp/443 for 
HTTPS... but it doesn't work. The problem is that the proposed API doesn't 
allow to specify a protocol factory per port. For such use case, you have to 
handle multiple servers.

Anyway, I still like the idea of listening to multiple hosts and/or multiple 
ports. It's a common requirement to start a server.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

About the function's signature to accept multiple hosts & ports, we could 

- accept host and port arguments being sequences and then we bind to all 
host:port combinations. Like if len(host) == N and len(port) == M, we bind to 
N*M sockets.

or

- accept host and port arguments being sequences and allow repetitions and bind 
to host:port couples from zip(host, port).

I think the first is "clean", but the second allows more flexibility.
Like for instance binding to IP1:port1 IP2:port2 but not IP1:port2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

> I can be convenient to have a single Server object with *all* listening 
> sockets. So I like the idea of supporting multiple ports. It's very cheap to 
> support it.

I like the idea too, but I'm not sure how to redesign the function's signature 
to accept multiple hosts & ports.  We have a separate param for 'port', and 
this patch allows to pass an array of hosts in 'host' argument.  Should we 
allow passing an array of (host, port) tuples in 'host' with 'None' in 'port'?  
This would be one ugly method...

> Since the consensus on python-dev looks more to keep the provisional API for 
> asyncion in Python 3.5, I propose to even modify Python 3.4 to not add too 
> many "if python >= 3.5..." checks in asyncio source code.

asyncio is provisional in 3.4 (and now in 3.5 too), so we can just keep 
updating it in bugfix releases.

> If you really prefer to keep Python 3.4 unchanged, we should at least add the 
> feature in Python 3.5.1, having to wait for Python 3.6 is too long, no?

This was before we reinstated asyncio as provisional in 3.5.

But before committing this change, let's agree if we want this API to accept 
multiple ports when you specify more than one host.

We also need to create a dev branch on github/asyncio, so that it's easier for 
us to sync PyPI package with CPython releases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

Since the consensus on python-dev looks more to keep the provisional API for 
asyncion in Python 3.5, I propose to even modify Python 3.4 to not add too many 
"if python >= 3.5..." checks in asyncio source code.

If you really prefer to keep Python 3.4 unchanged, we should at least add the 
feature in Python 3.5.1, having to wait for Python 3.6 is too long, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

> The only remaining question is is it OK that we can specify more than one 
> host but only one port?  Guido, Victor?

I can be convenient to have a single Server object with *all* listening 
sockets. So I like the idea of supporting multiple ports. It's very cheap to 
support it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

A new patch with capitalization of first words of sentences + splitting lines 
which are too long (more PEP8 friendly).

--
Added file: http://bugs.python.org/file40280/multibind4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

The traditional solution is to rewrite the sentence so the name is not the 
first word. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

About the missing capitalization, I omitted it because it's the argument's 
name. But I can resubmit with it being capitalized.
I forgot to run flake8, sorry!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

It can't go into 3.5.1 (a bugfix release) because it is a new feature (however 
small), and asyncio is not provisional in 3.5 (only in 3.4).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

Also, I guess you'll need a branch that tracks what's going into 3.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

> Yeah, I think this is fine. I think the use case for multiple hosts is very 
> different than that for multiple ports, so I see no reason to link the two 
> features, and multiple ports hasn't been requested.

OK!

> The patch has a long line and the new docstring looks a little odd (sentences 
> not starting with capital letters) 

Yeah, I've noticed that too, and made a mental note to check line lengths 
before committing.

> but otherwise I'm in favor of this going into 3.6.

Why only 3.6?  The change is fully backwards compatible, why can't it go in 
3.5.1?

Also, what's the protocol for committing something that might take years to be 
available in github/asyncio?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

Yeah, I think this is fine. I think the use case for multiple hosts is very 
different than that for multiple ports, so I see no reason to link the two 
features, and multiple ports hasn't been requested.

The patch has a long line and the new docstring looks a little odd (sentences 
not starting with capital letters) but otherwise I'm in favor of this going 
into 3.6.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

Thanks!

The only remaining question is is it OK that we can specify more than one host 
but only one port?  Guido, Victor?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

All right, here is the new patch then.

--
Added file: http://bugs.python.org/file40275/multibind3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

> How am I supposed to handle the method being called with a Mock'ed 
> getaddrinfo?

Why would you call it that way?  Are regular asyncio users going to mock it in 
their production/test code?

If the only case for using mocks there are asyncio unittests -- just don't do 
that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

All right, thanks.
I fixed everything but the thing about getaddrinfo being Mock'ed.
How am I supposed to handle the method being called with a Mock'ed getaddrinfo?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

Left some feedback in the code review.

I have a question about the api: so we allow to pass a set of hosts to 
create_server, but why do we have only a single port?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

Ah yes, sorry about that.
Here is a new patch.
However, I worked on these sources: https://github.com/python/asyncio which now 
I understand was not the correct repository, but I added a 
tests_require="netifaces" in the setup.py, where can I add this dependency in 
the CPython repository architecture?
Thanks!

--
Added file: http://bugs.python.org/file40271/multibind2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Changes by Yann Sionneau :


Removed file: http://bugs.python.org/file40270/multibind2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yury Selivanov]

Yury Selivanov added the comment:

> Here is a new patch, with modifications according to your review Victor.

Could you please generate the patch with 'hg diff', so that code review works 
with it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

Here is a new patch, with modifications according to your review Victor.
Thanks!

--
Added file: http://bugs.python.org/file40270/multibind2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

I reviewed multibind.patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

ping ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

Please let me know if you need anything for this patch to be integrated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

Could use a test. Hopefully Victor can do a thorough review.

On Tue, Mar 10, 2015 at 12:20 PM, Sebastien Bourdeauducq <
rep...@bugs.python.org> wrote:

>
> Sebastien Bourdeauducq added the comment:
>
> See attached.
>
> --
> keywords: +patch
> Added file: http://bugs.python.org/file38428/asyncio_multibind.diff
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Sebastien Bourdeauducq]

New submission from Sebastien Bourdeauducq:

I would like to be able to bind asyncio TCP servers to an arbitrary list of 
hosts, not just either one host or all interfaces.

I propose that the host parameter of create_server and start_server can be a 
list of strings that describes each host. Sockets are created for the set of 
all addresses of all specified hosts. The list may also contain None, or the 
empty string, in which case all interfaces are assumed.

If a string or None is passed directly, the behavior is unchanged - so this 
should not break compatibility.

I can submit a patch if this feature is approved.

--
components: asyncio
messages: 237791
nosy: gvanrossum, haypo, sebastien.bourdeauducq, yselivanov
priority: normal
severity: normal
status: open
title: support multiple hosts in create_server/start_server
type: enhancement
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[STINNER Victor]

STINNER Victor added the comment:

Combine patch written for CPython tree, so we get the "review" button.

--
Added file: http://bugs.python.org/file38442/multibind.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Yann Sionneau]

Yann Sionneau added the comment:

Here is a test related to Sebastien's patch.

--
nosy: +ysionneau
Added file: http://bugs.python.org/file38441/asyncio_multibind_test.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Sebastien Bourdeauducq]

Sebastien Bourdeauducq added the comment:

See attached.

--
keywords: +patch
Added file: http://bugs.python.org/file38428/asyncio_multibind.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

So it sounds like you already have a patch in mind... Can you work on it and 
upload it? (It's open source -- you get to make it yourself. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Sebastien Bourdeauducq]

Sebastien Bourdeauducq added the comment:

That could work, but I would have to manually resolve and filter the lists of 
hostnames so that the program does not attempt to bind the same address twice - 
something that is better implemented in create_server, which already does the 
hostname resolution. Additionally, the asyncio Server internally supports 
listening on multiple sockets, and that would be the right feature to use for 
implementing this instead of creating and managing multiple servers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23630] support multiple hosts in create_server/start_server

[Guido van Rossum]

Guido van Rossum added the comment:

This makes some sense, but it's easy to work around -- just call 
create_server() multiple times, once for each host. Why does that not work for 
you?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com