[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Donald Stufft]

Changes by Donald Stufft :


--
pull_requests: +883

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Berker Peksag]

Berker Peksag added the comment:

Thank you Gergely and Markus!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Roundup Robot]

Roundup Robot added the comment:

New changeset a5e8fe666c6b by Berker Peksag in branch '3.5':
Issue #25895: Enable WebSocket URL schemes in urllib.parse.urljoin
https://hg.python.org/cpython/rev/a5e8fe666c6b

New changeset 9bf370a33938 by Berker Peksag in branch '3.6':
Issue #25895: Merge from 3.5
https://hg.python.org/cpython/rev/9bf370a33938

New changeset 2fc89d6eff20 by Berker Peksag in branch 'default':
Issue #25895: Merge from 3.6
https://hg.python.org/cpython/rev/2fc89d6eff20

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Markus Holtermann added the comment:

Thanks for your input. I remove the versionchanged block.

--
Added file: 
http://bugs.python.org/file44649/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[R. David Murray]

R. David Murray added the comment:

I believe our stance on these has changed over time from "feature" to "bugfix", 
especially when there is a standards document to back it (but sometimes when 
there is not...we gave up a while ago on the MIME-type standards ever getting 
finalized and just bow to the reality of something being a defacto standard).

Given that, there should be no versionchanged directive.  I agree that if you 
want a versionchanged, then it is an enhancement and can only go into 3.7.

I'm a bit conflicted about whether these are bug fixes or features...I used to 
be on the feature side, but it is really annoying to have these 
"standard" things not work, so I'm more on the bugfix side now.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Martin Panter]

Martin Panter added the comment:

IMO if a versionadded/versionchanged notice is relevant, that is a good sign it 
is a feature rather than bug fix. If the 3.6.1 documentation says “added in 
3.5.3”, how do you know if it is in 3.6.0?

I looked at the history of other schemes being added to try and understand if 
it is a bug fix or not. I guess it is not that clear. In Python 2 the sftp and 
sips schemes are documented as a new feature in 2.5. Is that why you aren’t 
proposing any change to 2.7?

imap (Issue 618024), mms (7ec771893410): 2.3 features
rsync (Issue 981299): 2.3 bugfix (3a7e34dc6ae2)
svn: 2.4 bugfix (349a0fd598e0)
sftp (Issue 1407902): 2.5 feature (r42108); 2.4 bugfix reverted (r42123); 
documented (r42125) with version (r42158)
sips (r43520): 2.5 feature; documented with version
nfs (Issue 4962): 2.7 (r70757), 3.1 (r70760) features; 2.6 bugfix (r81131); 
undocumented
git (Issue 8657): 2.6, 2.7, 3.1 bugfixes; undocumented
tel (Issue 16713): 2.7, 3.2, 3.3 bugfixes; undocumented

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Robert Collins]

Robert Collins added the comment:

I find details like this extremely useful in the main docs, so please do add 
there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Markus Holtermann added the comment:

Thanks for your input, Berker. Updated as suggested. I still include the 
versionchanged annotation as I suspect more people to look at the docs than the 
entire changelog

--
Added file: 
http://bugs.python.org/file44615/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Berker Peksag]

Berker Peksag added the comment:

I think a Misc/NEWS entry is enough. We didn't use versionadded/versionchanged 
directives for similar changes in the past (mimetypes.types_map for example) 
Even if we decide to add an annotation it should be versionchanged, not 
versionadded (the API is not new.)

The rest of the patch looks good to me.

--
nosy: +berker.peksag
versions: +Python 3.6, Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Changes by Markus Holtermann :


Removed file: 
http://bugs.python.org/file44609/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.3.6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Changes by Markus Holtermann :


Removed file: 
http://bugs.python.org/file44610/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.master.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Yury Selivanov]

Changes by Yury Selivanov :


--
nosy:  -yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Markus Holtermann added the comment:

Since the patch applies cleanly to the 3.5, 3.6 and master branch I only 
attached one updated version of it.

--
Added file: 
http://bugs.python.org/file44612/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Changes by Markus Holtermann :


Removed file: 
http://bugs.python.org/file44608/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.3.5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Changes by Markus Holtermann :


Added file: 
http://bugs.python.org/file44609/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.3.6.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Changes by Markus Holtermann :


Added file: 
http://bugs.python.org/file44610/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.master.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Changes by Markus Holtermann :


Added file: 
http://bugs.python.org/file44608/0001-Enable-WebSocket-URL-schemes-in-urllib.parse.urljoin.3.5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Markus Holtermann]

Markus Holtermann added the comment:

As discussed with rbcollins during the KiwiPyCon sprints, I'll add patches for 
3.5, 3.6 and master with docs mentioning the addition of `ws` and `wss`  as of 
3.5.3

--
nosy: +MarkusH

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Robert Collins]

Robert Collins added the comment:

It is a bugfix, but we should document when it started working using the 
version added segment IMO. I think addding this to 3.6 now would be ok.

--
nosy: +rbcollins

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Martin Panter]

Martin Panter added the comment:

My view is that because the schemes are not in the documented list of supported 
schemes, this is a new feature, and any documentation update should include a 
“versionadded” or similar notice.

--
components: +Library (Lib)
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Yury Selivanov]

Yury Selivanov added the comment:

> The documentation should probably also be updated. It has a list of supported 
> schemes. 

I agree. Gergely, do you want to update your patch?

> I imagine this would have to be taken as a new feature for 3.6+, rather than 
> a bug fix.

Since urlparse can parse wss:// addresses, *but* urljoin can't work with them, 
I think we can classify this as a bug fix patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Martin Panter]

Martin Panter added the comment:

The documentation should probably also be updated. It has a list of supported 
schemes. I imagine this would have to be taken as a new feature for 3.6+, 
rather than a bug fix.

An alternative would be to enable joining for arbitrary schemes; see Issue 
18828.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Yury Selivanov]

Yury Selivanov added the comment:

Gergely, the patch looks alright. Please sign the contributor agreement and 
we'll have it merged.

--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Gergely Imreh]

Gergely Imreh added the comment:

Just signed it, thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25895] urllib.parse.urljoin does not handle WebSocket URLs

[Gergely Imreh]

New submission from Gergely Imreh:

Handling the "ws" and "wss" schemes of the WebSocket protocol in 
urllib.parse.urlparse was discussed in a previous issue 
http://bugs.python.org/issue13244 Parsing them seems to work correctly:

>>> urllib.parse.urlparse("wss://domain/endpoint")
ParseResult(scheme='wss', netloc='domain', path='/endpoint', params='', 
query='', fragment='')

On the other hand, urllib.parse.urljoin does not know about these schemes and 
thus cannot be used to construct WebSocket URLs (as, for example some streaming 
data APIs now seem to use it, such as 
https://starfighter.readme.io/docs/quotes-ticker-tape-websocket).

Using the example above I get:

>>> urllib.parse.urljoin("wss://domain/","/endpoint")
'/endpoint'

This is not the expected result, 'wss://example.com/endpoint', because these 
WebSocket schemes are not included in the uses_relative and uses_netloc lists 
in the library settings. For clarity, based on the previous issue's discussion 
and RFC6455 Section 11.1 https://tools.ietf.org/html/rfc6455#section-11.1 , 
these are the only two portions that apply, and should not be listed in 
uses_params.

As a workaround, similarly to the previous issue's recommendation, one can use:

>> import urlparse.parse
>> wsschemes = ["ws", "wss"]
>> urlparse.parse.uses_relative.extend(wsschemes)
>> urlparse.parse.uses_netloc.extend(wsschemes)

Attached patch to this effect. Also added tests to cover these schemes, though 
comments are definitely welcome on all of this!

--
files: 0001-urllib.parse-enable-WebSocket-URL-schemes-to-be-corr.patch
keywords: patch
messages: 256581
nosy: imrehg
priority: normal
severity: normal
status: open
title: urllib.parse.urljoin does not handle WebSocket URLs
type: behavior
versions: Python 3.4, Python 3.5
Added file: 
http://bugs.python.org/file41334/0001-urllib.parse-enable-WebSocket-URL-schemes-to-be-corr.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com