Re: [Python-Dev] Hash randomization for which types?
Stephen J. Turnbull wrote: > Glenn Linderman writes: > > > I think hashes of all types have been randomized, not _just_ the list > > you mentioned. > > Yes. There's only one hash function used, which operates on byte > streams IIRC. That function now has a random offset. The details of > hashing each type are in the serializations to byte streams. Could you please elaborate? Numbers are not hashed as byte streams, at least not up to Python 3.5. I am quite familiar with the way hashing of numbers is done in Python 2 & 3. (I had to re-implement this for a project of mine: https://pypi.python.org/pypi/tinyarray/) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wordcode v2
Demur Rumed gmail.com> writes: > I've personally benchmarked this fork with positive results. I'm skeptical of claims like this. What did you benchmark exactly, and with which results? I don't think changing the opcode encoding per se will bring any large benefit... Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hash randomization for which types?
On 02/16/2016 09:22 PM, Stephen J. Turnbull wrote: Glenn Linderman writes: > I think hashes of all types have been randomized, not _just_ the list > you mentioned. Yes. There's only one hash function used, which operates on byte streams IIRC. That function now has a random offset. The details of hashing each type are in the serializations to byte streams. Both these statements are wrong. int objects have their own hash algorithm, built in to long_hash() in Objects/longobject.c. The hash of an int is the value of the int, unless it's -1 or doesn't fit into the native type. And ints don't participate in hash randomization. //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hash randomization for which types?
Christoph Groth writes: > Stephen J. Turnbull wrote: > > Yes. There's only one hash function used, which operates on byte > > streams IIRC. That function now has a random offset. The details of > > hashing each type are in the serializations to byte streams. > > Could you please elaborate? Numbers are not hashed as byte streams, Just a stupid mistake on my part. Should have reviewed the code first. I'll shut up now, take my fly meds, get some sleep, drink coffee in the morning, and then take an axe to my keyboard. :-( Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hash randomization for which types?
On Thu, Feb 18, 2016 at 12:29 AM, Larry Hastings wrote: > int objects have their own hash algorithm, built in to long_hash() in > Objects/longobject.c. The hash of an int is the value of the int, unless > it's -1 or doesn't fit into the native type. Can someone elaborate on this special case, please? I can see the code there, but there's no comment. Is there some value in not hashing to -1? ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hash randomization for which types?
On 02/17/2016 08:49 AM, Chris Angelico wrote: On Thu, Feb 18, 2016 at 12:29 AM, Larry Hastings wrote: int objects have their own hash algorithm, built in to long_hash() in Objects/longobject.c. The hash of an int is the value of the int, unless it's -1 or doesn't fit into the native type. Can someone elaborate on this special case, please? I can see the code there, but there's no comment. Is there some value in not hashing to -1? Returning -1 indicates an error / exception. So hash functions never return -1 as a hash value. //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Hash randomization for which types?
Steven D'Aprano wrote: > On Tue, Feb 16, 2016 at 11:56:55AM -0800, Glenn Linderman wrote: >> On 2/16/2016 1:48 AM, Christoph Groth wrote: >> >Recent Python versions randomize the hashes of str, bytes and datetime >> >objects. I suppose that the choice of these three types is the result >> >of a compromise. Has this been discussed somewhere publicly? >> >> Search archives of this list... it was discussed at length. > > There's a lot of discussion on the mailing list. I think that this is > the very start of it, in Dec 2011: > (...) I tried searching myself for an hour or so, but though I found many discussions, I didn't see any discussion about whether hashes of other types should be randomized as well. The relevant PEP also doesn't touch this issue. > My recollection is that it was decided that only strings and bytes need > to have their hashes randomized, because only strings and bytes can be > used directly from user-input without first having a conversion step > with likely input range validation. In addition, changing the hash for > ints would break too much code for too little benefit: unlike strings, > where hash collision attacks on web apps are proven and easy, hash > collision attacks based on ints are more difficult and rare. > > See also the comment here: > > http://bugs.python.org/issue13703#msg151847 Perfect, that's exactly what I was looking for. I am reassured that this has been thought through. Thanks a lot! Christoph ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
On Tue, Feb 16, 2016 at 9:42 PM, Gregory P. Smith wrote: > > On Tue, Feb 16, 2016 at 9:00 PM Mike Kaplinskiy > wrote: > >> Hey folks, >> >> I hope this is the right list for this sort of thing (python-ideas seemed >> more far-fetched). >> >> For some context: there is currently a issue with pex that causes >> sys.modules lookups to stop working for __main__. In turns this makes >> unittest.run() & pkg_resources.resource_* fail. The root cause is that pex >> uses runpy.run_module with alter_sys=False. The fix should be to just pass >> alter_sys=True, but that changes sys.argv[0] and various existing pex files >> depend on that being the pex file. You can read more at >> https://github.com/pantsbuild/pex/pull/211 . >> >> Conservatively, I'd like to propose adding an argument to disable this >> behavior. The current behavior breaks a somewhat reasonable invariant that >> you can restart your program via `os.execv([sys.executable] + sys.argv)`. >> > > I don't know enough about pex to really dig into what it is trying to do > so this is tangential to answering your question but: > Sorry about that - a pex file is kind of like a relocatable virtualenv in one zip file. When it runs, it first executes some pex-specific code to extract packages (.egg, .whl) and add them to sys.path before running the actual user code. It's conceptually similar to a fat .jar file in JVM projects - all you need is `python`/`java` and all the code is in one file. > sys.executable may be None. ex: If you're an embedded Python interpreter > there is no Python executable. It cannot be blindly used re-execute the > current process. > > sys.argv represents the C main() argv array. Your inclination (in the > linked to bug above) to leave sys.argv[0] alone is a good one. > I was originally going to argue for getting rid of the feature entirely, but if runpy is to live up to the promise of being exactly the same as `python -m XXX yyy zzz`, it needs to be there. IMO it's bad form to depend on sys.argv[0] for anything but presentation purposes - usage messages and the like. It's hard to justify breaking compatibility for that though - unfortunately the runpy interface isn't pliable enough to really reimplement or unimplement this feature, and doing it by hand is...painful. You can also make an argument that `python runmodule.py module a b` and `python -m module a b` should be _exactly_ the same output, especially if `runmodule.py` is implementing something pass-through like profiling, coverage or tracing. A nicer interface might be some sort of callback to "do whatever you want before the module is executed", but that might be overkill. > > -gps > > Moreover it might be user-friendly to add a `argv=sys.argv[1:]` argument >> to set & restore the full arguments to the module, where `argv=None` >> disables argv[0] switching. >> >> What do you think? >> >> Mike. >> >> ___ >> Python-Dev mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org > > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Disabling changing sys.argv[0] with runpy.run_module(...alter_sys=True)
On Tue, 16 Feb 2016 at 20:59 Mike Kaplinskiy wrote: > Hey folks, > > I hope this is the right list for this sort of thing (python-ideas seemed > more far-fetched). > > For some context: there is currently a issue with pex that causes > sys.modules lookups to stop working for __main__. In turns this makes > unittest.run() & pkg_resources.resource_* fail. The root cause is that pex > uses runpy.run_module with alter_sys=False. The fix should be to just pass > alter_sys=True, but that changes sys.argv[0] and various existing pex files > depend on that being the pex file. You can read more at > https://github.com/pantsbuild/pex/pull/211 . > > Conservatively, I'd like to propose adding an argument to disable this > behavior. The current behavior breaks a somewhat reasonable invariant that > you can restart your program via `os.execv([sys.executable] + sys.argv)`. > Moreover it might be user-friendly to add a `argv=sys.argv[1:]` argument to > set & restore the full arguments to the module, where `argv=None` disables > argv[0] switching. > > What do you think? > This probably is best served as a feature request on bugs.python.org since it's not asking for some massive change or new feature but just a minor tweak to a module. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Buffer overflow bug in GNU C's getaddrinfo()
Is this something that we need to worry about? Extremely severe bug leaves dizzying number of software and devices vulnerable http://arstechnica.com/security/2016/02/extremely-severe-bug-leaves-dizzying-number-of-apps-and-devices-vulnerable/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buffer overflow bug in GNU C's getaddrinfo()
On Feb 17, 2016, at 10:44, MRAB wrote: > > Is this something that we need to worry about? > > Extremely severe bug leaves dizzying number of software and devices vulnerable > http://arstechnica.com/security/2016/02/extremely-severe-bug-leaves-dizzying-number-of-apps-and-devices-vulnerable/ Is there a workaround that Python and/or Python apps should be doing, or is this just a matter of everyone on glibc 2.9+ needs to update their glibc? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buffer overflow bug in GNU C's getaddrinfo()
Does python.org serve any Python binaries that are statically linked with a vulnerable glibc? That seems to be the question. If not, it's up to the downstream distributions. On Wed, Feb 17, 2016 at 12:09 PM, Andrew Barnert via Python-Dev wrote: > On Feb 17, 2016, at 10:44, MRAB wrote: >> >> Is this something that we need to worry about? >> >> Extremely severe bug leaves dizzying number of software and devices >> vulnerable >> http://arstechnica.com/security/2016/02/extremely-severe-bug-leaves-dizzying-number-of-apps-and-devices-vulnerable/ > > Is there a workaround that Python and/or Python apps should be doing, or is > this just a matter of everyone on glibc 2.9+ needs to update their glibc? > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buffer overflow bug in GNU C's getaddrinfo()
On Wed, Feb 17, 2016 at 12:12 PM Andrew Barnert via Python-Dev < [email protected]> wrote: > On Feb 17, 2016, at 10:44, MRAB wrote: > > > > Is this something that we need to worry about? > > > > Extremely severe bug leaves dizzying number of software and devices > vulnerable > > > http://arstechnica.com/security/2016/02/extremely-severe-bug-leaves-dizzying-number-of-apps-and-devices-vulnerable/ > > Is there a workaround that Python and/or Python apps should be doing, or > is this just a matter of everyone on glibc 2.9+ needs to update their glibc? > There are no workarounds that we could put within Python. People need to update their glibc and reboot. All *useful(*)* Linux distros have already released update packages. All of the infrastructure running Linux needs the update applied and a reboot (I'm guessing our infrastructure peeps have already done that). But this also includes Linux buildbots run by our random set of buildbot donors. -gps (*) off topic: Raspbian Wheezy is apparently not on the useful list. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] When does `PyType_Type.tp_alloc get assigned to PyType_GenericAlloc ?
On Sun, Feb 7, 2016 at 8:45 PM, Guido van Rossum wrote:
> I think it's probably line 2649 in typeobject.c, in type_new():
>
> type->tp_alloc = PyType_GenericAlloc;
I pondered it but it doesn't seem to be that. Isn't `type_new` called
*after* PyType_Type.tp_alloc has been called? I thought that line was only
being executed for user-defined types, and maybe some built-in types, but
certainly not PyType_Type.
On Sun, Feb 7, 2016 at 9:27 PM, eryk sun wrote:
> On Sun, Feb 7, 2016 at 7:58 AM, Randy Eels wrote:
> >
> > Yet, I can't seem to understand where and when does the `tp_alloc` slot
> of
> > PyType_Type get re-assigned to PyType_GenericAlloc. Does that even
> happen?
> > Or am I missing something bigger?
>
> _Py_InitializeEx_Private in Python/pylifecycle.c calls _Py_ReadyTypes
> in Objects/object.c. This calls PyType_Ready(&PyType_Type) in
> Objects/typeobject.c, which assigns type->tp_base = &PyBaseObject_Type
> and then calls inherit_slots. This executes COPYSLOT(tp_alloc), which
> assigns PyType_Type.tp_alloc = PyBaseObject_Type.tp_alloc, which is
> statically assigned as PyType_GenericAlloc.
>
> Debug trace on Windows:
>
> 0:000> bp python35!PyType_Ready
> 0:000> g
> Breakpoint 0 hit
> python35!PyType_Ready:
> `6502d160 4053pushrbx
> 0:000> ?? ((PyTypeObject *)@rcx)->tp_name
> char * 0x`650e4044
> "object"
> 0:000> g
> Breakpoint 0 hit
> python35!PyType_Ready:
> `6502d160 4053pushrbx
> 0:000> ?? ((PyTypeObject *)@rcx)->tp_name
> char * 0x`651d8e5c
> "type"
> 0:000> bp python35!inherit_slots
> 0:000> g
> Breakpoint 1 hit
> python35!inherit_slots:
> `6502c440 48895c2408 mov qword ptr [rsp+8],rbx
> ss:`0028f960={
> python35!PyType_Type
> (`6527cba0)}
>
> At entry to inherit_slots, PyType_Type.tp_alloc is NULL:
>
> 0:000> ?? python35!PyType_Type.tp_alloc
> * 0x`
> 0:000> pt
> python35!inherit_slots+0xd17:
> `6502d157 c3 ret
>
> At exit it's set to PyType_GenericAlloc:
>
> 0:000> ?? python35!PyType_Type.tp_alloc
> * 0x`65025580
> 0:000> ln 65025580
> (`65025580) python35!PyType_GenericAlloc |
> (`650256a0) python35!PyType_GenericNew
> Exact matches:
> python35!PyType_GenericAlloc (void)
>
This makes quite a bit of sense. I completely overlooked the interpreter
init routines.
Thank you both Guido and eryk!
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] RE 25939 - _ssl.enum_certificates broken on Windows
I've run into issue 25939 (https://bugs.python.org/issue25939) when trying to deploy a python webapp with IIS on Windows. This issue is preventing us from deploying the app to production as the workaround AFAICT requires running the app under an admin account. Apologies if this is an inappropriate forum for a +1 but I just wanted to let the devs know that this is an issue which affects the use of Python (on Windows) in the enterprise. I noticed that the patch hasn't been merged yet so was interested in making sure it didn't fall by the wayside... As a mere user I don't expect the devs to prioritize my own problems which no doubt only affect a very small number of python users but I would be very grateful if the patch did make it into a minor release. Regards, Dave ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
