Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-23 Thread eryk sun
On 3/23/19, Cameron Simpson wrote: > > Also, the common examples are attackers who are not the user making the > tempfile, in which case the _default_ mktemp is sort of secure with the > above because it gets made in /tmp which on a modern POSIX system > prevents _other_ uses from removing/renamin

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-23 Thread Cameron Simpson
On 20Mar2019 12:53, Jeroen Demeyer wrote: On 2019-03-20 12:45, Victor Stinner wrote: You can watch the /tmp directory using inotify and "discover" immediately the "secret" filename, it doesn't depend on the amount of entropy used to generate the filename. That's not the problem. The security

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-21 Thread eryk sun
On 3/20/19, Greg Ewing wrote: > Antoine Pitrou wrote: > >> How is it more secure than using mktemp()? > > It's not, but it solves the problem someone suggested of another > program not being able to access and/or delete the file. NamedTemporaryFile(delete=False) is more secure than naive use of m

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Greg Ewing
Antoine Pitrou wrote: On Wed, 20 Mar 2019 11:25:53 +1300 Greg Ewing wrote: So use NamedTemporaryFile(delete = False) and close it before passing it to the other program. How is it more secure than using mktemp()? It's not, but it solves the problem someone suggested of another program not

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Ivan Pozdeev via Python-Dev
Before we can say if something is "secure" or not, we need a threat model -- i.e we need to agree which use cases we are protecting and from what threats. So far, I've seen these use cases: 1. File for the current process' private use 2. File/file name generated by the current process; written

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Antoine Pitrou
On Wed, 20 Mar 2019 11:25:53 +1300 Greg Ewing wrote: > Antoine Pitrou wrote: > > Does it always work? According to the docs, """Whether the name can be > > used to open the file a second time, while the named temporary file is > > still open, varies across platforms > > So use NamedTemporaryFil

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Anders Munch
Victor Stinner: > To be clear: mktemp() is vulnerable by design No: mktemp() is vulnerable by implementation. Specifically, returning a file name in a world-accessible location, /tmp. regards, Anders ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Anders Munch
Steven D'Aprano: >> 128 bits seems like overkill: There's no birthday attack because >> no-one keeps 2^(ENTROPY_BITS/2) files around > You haven't seen my Downloads folder... :-) I put it to you that those files are not temporary :-) > Why be so miserly with entropy? I don't necessarily disagre

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Steven D'Aprano
On Wed, Mar 20, 2019 at 12:45:40PM +0100, Victor Stinner wrote: > Hi, > > I'm not really convinced that mktemp() should be made "more secure". > To be clear: mktemp() is vulnerable by design. It's not a matter of > entropy. You can watch the /tmp directory using inotify and "discover" > immediatel

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Jeroen Demeyer
On 2019-03-20 12:45, Victor Stinner wrote: You can watch the /tmp directory using inotify and "discover" immediately the "secret" filename, it doesn't depend on the amount of entropy used to generate the filename. That's not the problem. The security issue here is guessing the filename *before

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Steven D'Aprano
On Wed, Mar 20, 2019 at 11:25:03AM +, Anders Munch wrote: > 128 bits seems like overkill: There's no birthday attack because no-one keeps > 2^(ENTROPY_BITS/2) files around, You haven't seen my Downloads folder... :-) But seriously: > and the attack is running on the attackee's > system, so

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Victor Stinner
Hi, I'm not really convinced that mktemp() should be made "more secure". To be clear: mktemp() is vulnerable by design. It's not a matter of entropy. You can watch the /tmp directory using inotify and "discover" immediately the "secret" filename, it doesn't depend on the amount of entropy used to

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Anders Munch
Nathaniel J. Smith: > Historically, mktemp variants have caused *tons* of serious security > vulnerabilities. It's not a theoretical issue. All the more reason to have a standard library function that gets it right. > The choice of ENTROPY_BYTES is an interesting question. 16 (= 128 bits) would >

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread eryk sun
On 3/20/19, Anders Munch wrote: > > You are right, I must have mentally reversed the polarity of the delete > argument. And I didn't realise that the access right on a file had the > power to prevent itself from being removed from the folder that it's in. I > thought the access flags were a prop

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Anders Munch
Anders Munch: >>> So use NamedTemporaryFile(delete = False) and close it before passing it to >>> the other program. >> That's effectively the same as calling tempfile.mktemp. While it does >> waste time opening and closing an unused file, that doesn't help with >> security Sebastian Rittau: >

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Sebastian Rittau
Am 20.03.19 um 09:47 schrieb Anders Munch: Greg Ewing: So use NamedTemporaryFile(delete = False) and close it before passing it to the other program. That's effectively the same as calling tempfile.mktemp. While it does waste time opening and closing an unused file, that doesn't help with

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Anders Munch
Greg Ewing: > So use NamedTemporaryFile(delete = False) and close it before passing it to > the other program. That's effectively the same as calling tempfile.mktemp. While it does waste time opening and closing an unused file, that doesn't help with security. If anything, it might worsen se

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Serhiy Storchaka
19.03.19 15:39, Antoine Pitrou пише: The fact that many projects, including well-maintained ones such Sphinx or pip, use mktemp(), may be a hint that replacing it is not as easy as the people writing the Python documentation seem to think. Sorry, it was my mistake (searching mkdir instead of mk

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-20 Thread Serhiy Storchaka
19.03.19 16:21, Paul Ganssle пише: I'm not sure the relationship with mkdir and mktemp here. I don't see any uses of tempfile.mktemp in pip or setuptools, though they do use os.mkdir (which is not deprecated). Both pip and setuptools use pytest's tmpdir_factory.mktemp() in their test suites, but

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Nathaniel Smith
On Tue, Mar 19, 2019 at 3:43 PM Giampaolo Rodola' wrote: > > On Tue, Mar 19, 2019 at 6:21 PM Guido van Rossum wrote: > > >> On Tue, Mar 19, 2019 at 10:14 AM Giampaolo Rodola' > >> wrote: > >> Technically you cannot make it 100% safe, only less likely to occur. > > That seems unnecessarily pedan

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread eryk sun
On 3/19/19, Victor Stinner wrote: > > When I write tests, I don't really care of security, but > NamedTemporaryFile caused me many troubles on Windows: you cannot > delete a file if it's still open in a another program. It's way more > convenient to use tempfile.mktemp(). Opening the file again f

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Giampaolo Rodola'
On Tue, Mar 19, 2019 at 6:21 PM Guido van Rossum wrote: >> On Tue, Mar 19, 2019 at 10:14 AM Giampaolo Rodola' >> wrote: >> Technically you cannot make it 100% safe, only less likely to occur. > That seems unnecessarily pedantic. A cryptographic random generator, like the > one in the secrets m

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Greg Ewing
Antoine Pitrou wrote: Does it always work? According to the docs, """Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms So use NamedTemporaryFile(delete = False) and close it before passing it to the other program.

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Brett Cannon
On Tue, Mar 19, 2019 at 10:22 AM Guido van Rossum wrote: > On Tue, Mar 19, 2019 at 10:14 AM Giampaolo Rodola' > wrote: > >> >> On Tue, 19 Mar 2019 at 17:47, Sebastian Rittau >> wrote: >> >>> Am 19.03.19 um 17:23 schrieb Giampaolo Rodola': >>> > @Sebastian >>> >> If there are valid use cases for

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Sebastian Krause
Guido van Rossum wrote: > If all you need is a random name, why not just use a random number > generator? > E.g. I see code like this: > > binascii.hexlify(os.urandom(8)).decode('ascii') I tend to use os.path.join(dir, str(uuid.uuid4())) if I need to create a random filename to pass to anothe

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Guido van Rossum
On Tue, Mar 19, 2019 at 10:14 AM Giampaolo Rodola' wrote: > > On Tue, 19 Mar 2019 at 17:47, Sebastian Rittau wrote: > >> Am 19.03.19 um 17:23 schrieb Giampaolo Rodola': >> > @Sebastian >> >> If there are valid use cases for mktemp(), I recommend renaming >> >> it to mkname_unsafe() or something

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Giampaolo Rodola'
On Tue, 19 Mar 2019 at 17:47, Sebastian Rittau wrote: > Am 19.03.19 um 17:23 schrieb Giampaolo Rodola': > > @Sebastian > >> If there are valid use cases for mktemp(), I recommend renaming > >> it to mkname_unsafe() or something equally obvious. > > I'm -1 about adding an alias (there should be on

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Paul Moore
On Tue, 19 Mar 2019 at 16:47, Sebastian Rittau wrote: > But I had another thought: If I understand correctly, the exploitability > of mktemp() relies on the fact that between determining whether the > file exists and creation an attacker can create the file themselves. > Couldn't this problem be s

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Sebastian Rittau
Am 19.03.19 um 17:23 schrieb Giampaolo Rodola': @Sebastian If there are valid use cases for mktemp(), I recommend renaming it to mkname_unsafe() or something equally obvious. I'm -1 about adding an alias (there should be one and preferably only one way to do it). Also mkstemp() and mkdtemp() ar

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Giampaolo Rodola'
On Tue, Mar 19, 2019 at 3:57 PM Antoine Pitrou wrote: > > > Le 19/03/2019 à 15:52, Guido van Rossum a écrit : > > > > If all you need is a random name, why not just use a random number > > generator? > > E.g. I see code like this: > > > > binascii.hexlify(os.urandom(8)).decode('ascii') > > mkt

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Giampaolo Rodola'
On Tue, Mar 19, 2019 at 2:06 PM Stéphane Wirtel wrote: > > Hi, > > Context: raise a warning or remove tempfile.mktemp() > BPO: https://bugs.python.org/issue36309 > > Since 2.3, this function is deprecated in the documentation, just in the > documentation. In the code, there is a commented RuntimeW

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Anders Munch
Antoine Pitrou: > And if there is an easy replacement, then how about re-implementing > mktemp() using that replacement, instead of removing it? Indeed. The principal security issue with mktemp is the difficulty in creating a user-specific thing under a shared /tmp folder in a multi-user setup.

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Antoine Pitrou
On Tue, 19 Mar 2019 15:12:06 +0100 Sebastian Rittau wrote: > Am 19.03.19 um 14:53 schrieb Victor Stinner: > > > > When I write tests, I don't really care of security, but > > NamedTemporaryFile caused me many troubles on Windows: you cannot > > delete a file if it's still open in a another program

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Antoine Pitrou
Le 19/03/2019 à 15:52, Guido van Rossum a écrit : > > If all you need is a random name, why not just use a random number > generator? > E.g. I see code like this: > >     binascii.hexlify(os.urandom(8)).decode('ascii') mktemp() already does this, probably in a better way, including the notion o

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Guido van Rossum
On Tue, Mar 19, 2019 at 6:27 AM Antoine Pitrou wrote: > > -1. Please don't remove tempfile.mktemp(). mktemp() is useful to > create a temporary *name*. All other tempfile functions create an > actual file and impose additional burden, for example by making the > file unaccessible by other proc

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Antoine Pitrou
On Tue, 19 Mar 2019 15:10:40 +0100 Stéphane Wirtel wrote: > totally agree with you but this function is deprecated (2002) since 2.3, > with a simle comment about a security issue. "Deprecated" doesn't mean anything here. It's just a mention in the documentation. It doesn't produce actual warnin

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Paul Ganssle
I'm not sure the relationship with mkdir and mktemp here. I don't see any uses of tempfile.mktemp in pip or setuptools, though they do use os.mkdir (which is not deprecated). Both pip and setuptools use pytest's tmpdir_factory.mktemp() in their test suites, but I believe that is not the same thing

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Sebastian Rittau
Am 19.03.19 um 14:53 schrieb Victor Stinner: When I write tests, I don't really care of security, but NamedTemporaryFile caused me many troubles on Windows: you cannot delete a file if it's still open in a another program. It's way more convenient to use tempfile.mktemp(). O_EXCL, open(tmpname,

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Stéphane Wirtel
totally agree with you but this function is deprecated (2002) since 2.3, with a simle comment about a security issue. 2.3 -> 2.7, 3.0 -> 3.7, 13 releases and 17 years. Maybe we could remove it with an official PendingDeprecationWarning. Le 19/03/19 à 14:39, Antoine Pitrou a écrit : > The fact th

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Victor Stinner
Hi, I would prefer to keep tempfile.mktemp(), remove the deprecation, but better explain the risk of race condition affecting security. Le mar. 19 mars 2019 à 14:41, Chris Angelico a écrit : > Can't you create a NamedTemporaryFile and permit the other program to > use it? I just tried that (with

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Antoine Pitrou
On Wed, 20 Mar 2019 00:37:56 +1100 Chris Angelico wrote: > On Wed, Mar 20, 2019 at 12:25 AM Antoine Pitrou wrote: > > > > > > -1. Please don't remove tempfile.mktemp(). mktemp() is useful to > > create a temporary *name*. All other tempfile functions create an > > actual file and impose additi

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Antoine Pitrou
On Tue, 19 Mar 2019 15:32:25 +0200 Serhiy Storchaka wrote: > 19.03.19 15:03, Stéphane Wirtel пише: > > Suggestion and timeline: > > > > 3.8, we raise a PendingDeprecationWarning > > * update the code > > * update the documentation > > * update the tests > >(check a PendingD

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Chris Angelico
On Wed, Mar 20, 2019 at 12:25 AM Antoine Pitrou wrote: > > > -1. Please don't remove tempfile.mktemp(). mktemp() is useful to > create a temporary *name*. All other tempfile functions create an > actual file and impose additional burden, for example by making the > file unaccessible by other pr

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Serhiy Storchaka
19.03.19 15:03, Stéphane Wirtel пише: Suggestion and timeline: 3.8, we raise a PendingDeprecationWarning * update the code * update the documentation * update the tests (check a PendingDeprecationWarning if sys.version_info == 3.8) 3.9, we change PendingDeprecationWarning

Re: [Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Antoine Pitrou
-1. Please don't remove tempfile.mktemp(). mktemp() is useful to create a temporary *name*. All other tempfile functions create an actual file and impose additional burden, for example by making the file unaccessible by other processes. But sometimes all I want is a temporary name that an *oth

[Python-Dev] Remove tempfile.mktemp()

2019-03-19 Thread Stéphane Wirtel
Hi, Context: raise a warning or remove tempfile.mktemp() BPO: https://bugs.python.org/issue36309 Since 2.3, this function is deprecated in the documentation, just in the documentation. In the code, there is a commented RuntimeWarning. Commented by Guido in 2002, because the warning was too annoyi