[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Sviatoslav Sydorenko]

Sviatoslav Sydorenko  added the comment:

Not sure if it's in the scope of this issue but I thought I'd report it.

I've just hit a similar issue with `setup.cfg` under Python 3.8 which is 
supposed to be parsed by `setuptools` but for some reason, I only see 
`distutils` in the trace, coming from `pip`.

```
$ python -m pip install --upgrade tox
ERROR: Exception:
Traceback (most recent call last):
  File 
"/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/pip/_internal/cli/base_command.py",
 line 186, in _main
status = self.run(options, args)
  File 
"/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/pip/_internal/commands/install.py",
 line 253, in run
options.use_user_site = decide_user_install(
  File 
"/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/pip/_internal/commands/install.py",
 line 604, in decide_user_install
if site_packages_writable(root=root_path, isolated=isolated_mode):
  File 
"/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/pip/_internal/commands/install.py",
 line 549, in site_packages_writable
test_writable_dir(d) for d in set(get_lib_location_guesses(**kwargs))
  File 
"/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/pip/_internal/commands/install.py",
 line 543, in get_lib_location_guesses
scheme = distutils_scheme('', *args, **kwargs)
  File 
"/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/site-packages/pip/_internal/locations.py",
 line 109, in distutils_scheme
d.parse_config_files()
  File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/distutils/dist.py", 
line 413, in parse_config_files
val = parser.get(section,opt)
  File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/configparser.py", 
line 799, in get
return self._interpolation.before_get(self, section, option, value,
  File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/configparser.py", 
line 395, in before_get
self._interpolate_some(parser, option, L, value, section, defaults, 1)
  File "/opt/hostedtoolcache/Python/3.8.2/x64/lib/python3.8/configparser.py", 
line 442, in _interpolate_some
raise InterpolationSyntaxError(
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', 
found: '%3Adevel\nCode of Conduct = 
https://docs.ansible.com/ansible/latest/community/code_of_conduct.html\nSource 
Code = https://github.com/ansible/pylibssh'
```
(https://github.com/ansible/pylibssh/runs/640262804?check_suite_focus=true#step:7:32)

This is caused by
```
diff --git a/setup.cfg b/setup.cfg
index 9318235..0455376 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -9,7 +9,7 @@ version = 0.0.1.dev1
 url = https://github.com/ansible/pylibssh
 project_urls =
 Bug Tracker = https://github.com/ansible/pylibssh/issues
-CI: Travis = https://travis-ci.com/ansible/pylibssh
+CI: GitHub Workflows = 
https://github.com/ansible/pylibssh/actions?query=branch%3Adevel
 Code of Conduct = 
https://docs.ansible.com/ansible/latest/community/code_of_conduct.html
 Source Code = https://github.com/ansible/pylibssh
 description = Python bindings for libssh client specific to Ansible use case
```

--
nosy: +webknjaz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Thomas Levine]

Thomas Levine added the comment:

I just upgraded from 3.5.1 to 3.5.2 and found that I couldn't authenticate to 
PyPI anymore. And then I remembered that this had been fixed, so removed the 
extra percent sign, and my uploads worked. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Roundup Robot]

Roundup Robot added the comment:

New changeset 83d731f01dde by Berker Peksag in branch '3.5':
Issue #20120: Add a test case to verify the % char can be used in .pypirc
https://hg.python.org/cpython/rev/83d731f01dde

New changeset 703d9066c459 by Berker Peksag in branch 'default':
Issue #20120: Merge from 3.5
https://hg.python.org/cpython/rev/703d9066c459

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Jason R. Coombs]

Jason R. Coombs added the comment:

Fixed now. I'll watch the buildbots to assure acceptance.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Roundup Robot]

Roundup Robot added the comment:

New changeset eae59b6bf133 by Jason R. Coombs in branch '3.5':
Issue #20120: Use RawConfigParser for .pypirc parsing, removing support for 
interpolation unintentionally added with move to Python 3. Behavior no longer 
does any interpolation in .pypirc files, matching behavior in Python 2.7 and 
Setuptools 19.0.
https://hg.python.org/cpython/rev/eae59b6bf133

New changeset 89116bd505cb by Jason R. Coombs in branch 'default':
Issue #20120: Merge with 3.5
https://hg.python.org/cpython/rev/89116bd505cb

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Jason R. Coombs]

Jason R. Coombs added the comment:

I just confirmed only Python 3.5 gets bugfixes, so I'll apply this to 3.5 and 
default.

--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Łukasz Langa]

Łukasz Langa added the comment:

Oh, my bad. Yeah, that sounds like a worthwhile change.

--
resolution: not a bug -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Jason R. Coombs]

Jason R. Coombs added the comment:

Łukasz, my proposal was to also update distutils (in all Python versions that 
get bugfixes) to match the Setuptools behavior and Python 2 behavior. I'd still 
like to do that to harmonize the implementations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Łukasz Langa]

Łukasz Langa added the comment:

Closing this as setuptools switched to RawConfigParser.

--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[R. David Murray]

R. David Murray added the comment:

So fixing distutils to use RawConfigParser?  How likely is that to break 
currently working python3-only code?  I'm imagining from what you wrote that 
your answer is "very close to zero', but I'd like explicit confirmation :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Thomas Levine]

Thomas Levine added the comment:

I noticed something else that might be interesting.
If I delete my .pypirc and have setuptools create it,

  python3.5 -c 'import setuptools; setuptools.setup()' register

the .pypirc is created with the un-escaped percent sign and without error.
I receive the error when I try to use the .pypirc next time.

If a different config parser is being used in the writing, perhaps we could at 
least use the same one in both places.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Jason R. Coombs]

Jason R. Coombs added the comment:

Setuptools 19 has been out for over a month now with no adverse consequences. 
Łukasz and David, as you both originally tagged this as won't fix, how would 
you feel about interpreting this as a defect and regression over Python 2.7 and 
fixing it it all Python 3 versions that receive bug fixes and bringing it into 
parity with Setuptools? If you give me the green light, I'll apply the patches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Thomas Levine]

Thomas Levine added the comment:

The error message could just say. something to the effect of ".pypyrc works 
differently in Python 2 and Python 3. If you want a percent sign in Python 3, 
you have to escape it with another percent sign".

Suppose someone is already using a configuration file that specifies a value as 
having two literal percent signs in a row, like a password that actually 
contains two percent signs. I see no way of fixing the present bug without 
breaking this situation, and I presume that this is why Antoine suggested that 
this is a wont fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Jason R. Coombs]

Jason R. Coombs added the comment:

It doesn't make sense to give guidance to use one percent sign on one Python 
and two on another because .pypirc applies to all Python versions. I'm sure 
it's not uncommon for one to use different versions of Python for various 
package releases.

I'm planning to use RawConfigParser in Setuptools. If there's no fallout from 
this change, I suggest that Python 3 should do the same for distutils in bugfix 
releases, as this behavior is a regression over Python 2.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Jason R. Coombs]

Jason R. Coombs added the comment:

This same issue applies to setuptools 
(https://bitbucket.org/pypa/setuptools/issues/442/setuptools-1832-cannot-access-pypi-if-pypi).

I've created a test that captures the failure, but I struggled to find a 
solution that doesn't break across Python versions.

Thomas suggested guiding the user to provide two percent signs instead of one, 
but that breaks on Python 2, which will interpret the value as two percent 
signs.

As suggested, using RawConfigParser does address the issue on both Python 2 and 
3, but that of course could break compatibility if interpolation was expected.

I have found that if one uses SafeConfigParser on Python 2 and 3, and requires 
the user to use two percent signs to represent one, it does seem to work on 
both Python 2 and 3.

--
nosy: +jason.coombs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Laszlo Nagy]

Laszlo Nagy added the comment:

When I uploaded via "setup.py register" and created a new user, then distutils 
saved the password in the wrong format! So maybe the password read part is a 
wontfix, but then the save part must be fixed.

--
nosy: +nagylzs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Thomas Levine]

New submission from Thomas Levine:

This works fine in Python 2.7, but it fails in Python 3.3.

[tlevine@wildebeest mailfest-scoreboard]$ python3 --version
Python 3.3.3
[tlevine@wildebeest mailfest-scoreboard]$ python3 setup.py register sdist
running register
Traceback (most recent call last):
  File setup.py, line 11, in module
scripts=['scoreboard']
  File /usr/lib/python3.3/distutils/core.py, line 148, in setup
dist.run_commands()
  File /usr/lib/python3.3/distutils/dist.py, line 929, in run_commands
self.run_command(cmd)
  File /usr/lib/python3.3/distutils/dist.py, line 948, in run_command
cmd_obj.run()
  File /usr/lib/python3.3/distutils/command/register.py, line 45, in run
self._set_config()
  File /usr/lib/python3.3/distutils/command/register.py, line 71, in 
_set_config
config = self._read_pypirc()
  File /usr/lib/python3.3/distutils/config.py, line 83, in _read_pypirc
current[key] = config.get(server, key)
  File /usr/lib/python3.3/configparser.py, line 790, in get
d)
  File /usr/lib/python3.3/configparser.py, line 391, in before_get
self._interpolate_some(parser, option, L, value, section, defaults, 1)
  File /usr/lib/python3.3/configparser.py, line 440, in _interpolate_some
found: %r % (rest,))
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', 
found: '%rest-of-my-password'



Here are the relevant files.



[tlevine@wildebeest mailfest-scoreboard]$ cat ~/.pypirc 
[distutils]
index-servers =
pypi

[pypi]
username:tlevine
password:yh^%#rest-of-my-password
[tlevine@wildebeest mailfest-scoreboard]$ cat setup.py 
#!/usr/bin/env python3

from distutils.core import setup

setup(name='mailfest-scoreboard',
  version='0.0.1',
  description='Score mailfest participants',
  author='Thomas Levine',
  author_email='_...@thomaslevine.com',
  url='https://github.com/tlevine/mailfest-scoreboard',
  scripts=['scoreboard']
 )

--
components: Distutils
messages: 207275
nosy: tlevine
priority: normal
severity: normal
status: open
title: Percent-signs (%) in .pypirc should not be interpolated
type: behavior
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Thomas Levine]

Thomas Levine added the comment:

The relevant section  distutils/config.py seems no different in Python 2.7, so 
now I see this as a bug in configparser.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Thomas Levine]

Thomas Levine added the comment:

Hmm now it looks to me like this is the intended behavior.
http://hg.python.org/cpython/file/ea0aa3e32ab5/Lib/test/test_configparser.py#l1541

Switching the single percent sign (%) to two (%%) in .pypirc makes it work. 
Maybe we can make a nicer error message though.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[R. David Murray]

R. David Murray added the comment:

It could be an issue of which configparser is/was used.

--
nosy: +lukasz.langa, r.david.murray
versions: +Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Antoine Pitrou]

Antoine Pitrou added the comment:

In 3.x, the ConfigParser class is 2.x's SafeConfigParser, and the parsing rules 
are different (stricter, it seems).

It's probably a won't fix, I'd say.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[R. David Murray]

R. David Murray added the comment:

More rational is probably closer to the truth :).

Yeah, it's probably a won't fix, but it would be nice to have Łukasz's input.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20120] Percent-signs (%) in .pypirc should not be interpolated

[Łukasz Langa]

Łukasz Langa added the comment:

Yes, that's wontfix but we can make the error message more friendly so I'll 
leave this open.

The .pypirc parser should have been RawConfigParser from the start, the 
interpolation mechanism is obscure enough it's probably not used. Unfortunately 
this probably means we cannot easily switch it to RawConfigParser now.

--
assignee:  - lukasz.langa
versions: +Python 3.5 -Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20120
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com