Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 11:31 PM Tim Peters wrote: > > The best approach is indeed to pass the function pointer to every > location that needs it. Note that a MergeState struct is already > created per sort invocation, That isn't a file global for much the > same reason. >

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Mon, Mar 6, 2017 at 5:52 PM, Tim Peters wrote: > [Chris Angelico ] >> Arbitrary comparison functions let you do anything but whoa, I >> cannot imagine any way that this would ever happen outside of "hey >> look, here's how you can trigger a

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

[Chris Angelico ] > Arbitrary comparison functions let you do anything but whoa, I > cannot imagine any way that this would ever happen outside of "hey > look, here's how you can trigger a SystemError"! CPython is full of defensive code protecting against malicious crap.

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 11:39 PM INADA Naoki wrote: > > I think there is another safe solution: Gave up unsafe_object_compare. > > Compare function of long, float, and unicode must not call list.sort(), > and must not release GIL. > So all you need is, backup old

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

2017-03-05 22:08 GMT-08:00 Elliot Gorokhovsky : > Another solution: check if there is more than one thread; if there is, then > disable optimization. Is sorting in multithreaded programs common enough to > warrant adding the complexity to deal with it? > I think

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 10:45 PM Elliot Gorokhovsky < elliot.gorokhov...@gmail.com> wrote: > > the problem is, how can we get a unique identifier for the thread in a > platform-independent way? Any ideas? > Oh, I could probably just copy over code from threading.get_ident()... not sure if the

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 10:25 PM David Mertz wrote: > > Could we make the file global a table of comparison functions and have > each thread reference a position in the table? It would be fine if multiple > threads happened to use the position of e.g. the int comparison, just so

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 9:25 PM Tim Peters wrote: > > Would someone please move the patch along? I expect it's my fault it's > languished so long, since I'm probably the natural person to review it, but > I've been buried under other stuff. > > But the patch doesn't change

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

[Elliot Gorokhovsky ] > (Summary of results: my patch at https://bugs.python.org/issue28685 makes list.sort() 30-50% > faster in common cases, and at most 1.5% slower in the uncommon worst case.) > ... Would someone please move the patch along? I expect it's my

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 9:12 PM MRAB wrote: > > Although it's true that both programmers and Python might treat 10 as > functionally identical to 10.0, in practice the numbers that are being > added to the list probably come from some code that returns integers > /or/

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On 2017-03-06 03:09, Chris Angelico wrote: On Mon, Mar 6, 2017 at 2:03 PM, Elliot Gorokhovsky wrote: On Sun, Mar 5, 2017 at 7:50 PM Chris Angelico wrote: I would be rather curious to know how frequently a list consists of "numbers", but a mix

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

I see the benchmarks, and while I assume the asymptotic complexity is the same, is there a longer "start-up" time your optimizations need? Do you have benchmarks where you sort 10, 100...10**6 items to show that beyond the types you're sorting, you're not amortizing any increased overhead out to

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 8:23 PM David Mertz wrote: > > But also, it's not just the number of list objects that are sorted, but > how often it's done. I could have a 10,000 line program with only one call > to `my_list.sort()` in it... but that one line is something that is

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 8:20 PM David Mertz wrote: > > B. I think a very large percentage of lists are heterogeneous. But most > of the time when they are, it's not because there are several different > numeric types but rather because a list collects some sort of custom >

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 8:10 PM Chris Angelico wrote: > > Also, the performance hit is so small, and even that is in the very > worst case (a homogeneous list with one different type at the end). I > like changes that make stuff run faster. > I agree.

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

In my experience and/or estimation... well two things: A. Mixtures of floats/ints seem a lot more common than the 10% being thrown around. I don't even consider that bad practice currently (it might become bad practice if Elliot's patch is used, since keeping elements float-only would allow much

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 6:45 PM, Steven D'Aprano wrote: > Here is a radical thought... why don't lists track their common type > themselves? There's only a few methods which can add items: > I had exactly the same thought. Lists would need to grow a new attribute, of

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 7:50 PM Chris Angelico wrote: > > I would be rather curious to know how frequently a list consists of > "numbers", but a mix of ints and floats. Does it happen a > lot in real-world code? > > This is of course undecidable to verify statically, so we can't

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

This is exactly what I would have done if I didn't think the changes would be too large-scale to ever be accepted (at least if proposed by someone without any experience). This is also what my friend suggested when I explained my project to him. In fact, this is exactly what dictionaries do: at

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Mon, Mar 6, 2017 at 12:53 PM, Elliot Gorokhovsky wrote: > (1) lists are idiomatically type-homogeneous. To quote the Python > documentation, "Lists are mutable, and *their elements are usually > homogeneous* and are accessed by iterating over the list". > (2) it's

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 05, 2017 at 07:19:43AM +, Elliot Gorokhovsky wrote: > You may remember seeing some messages on here about optimizing list.sort() > by exploiting type-homogeneity: since comparing apples and oranges is > uncommon (though possible, i.e. float to int), it pays off to check if the >

Re: [Python-ideas] get() method for list and tuples

On 03/05/2017 11:13 AM, Ed Kellett wrote: I'm not trying to get anyone to implement list.get, I'm trying to get it centrally > documented and allowed into list's overly-mappingproxied namespace. --> dir(list) # non dunder methods 'append', 'clear', 'copy', 'count', 'extend', 'index',

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 6:30 PM INADA Naoki wrote: > Good job. I'll read your work later. > Thanks! So please don't use string-key dict specialization to rationalize > list-sort specialization. > I'm not quite doing that: I agree that dict is a special case because of

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

Good job. I'll read your work later. > relatively simple optimization. I would also add that Python dictionaries > already implement this optimization: they start out optimizing based on the > assumption that they'll only be seeing string keys, checking to make sure > that assumption holds as

Re: [Python-ideas] get() method for list and tuples

On 5 March 2017 at 19:13, Ed Kellett wrote: >> I think we're going to have to just disagree. You won't convince me >> it's worth adding list.get unless you can demonstrate some *existing* >> costs that would be removed by adding list.get, and showing that they >> are greater

Re: [Python-ideas] get() method for list and tuples

On 05/03/2017 20:16, Ed Kellett wrote: On Sun, 5 Mar 2017 at 19:54 David Mertz > wrote: In terms of an actual use case, I can see it for "Lists no longer than 4". That's an excessively hard limit. Any other use of this hypothetical method

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sun, Mar 5, 2017 at 10:51 AM David Mertz wrote: > Thanks for the hard work, this looks very promising. > Thank you! > Real world data tends to be mostly-sorted. So it would be useful for your > benchmarks to include: > > A) Performance on completely sorted data > i) Of

Re: [Python-ideas] get() method for list and tuples

On Mon, Mar 6, 2017 at 6:13 AM, Ed Kellett wrote: > The point about this as a Python change is that it's a standard. Who does > the work couldn't be less relevant; what matters is that it would add a > consistent and easy spelling for something that doesn't have one. Oh,

Re: [Python-ideas] get() method for list and tuples

On Sun, Mar 5, 2017 at 11:22 AM, Ed Kellett wrote: > On Sun, 5 Mar 2017 at 18:13 David Mertz wrote: > >> data = args.get(17, "") >> >> Then I'm pretty sure the programmer thinks she's being passed a very >> different type of collection than is actually

Re: [Python-ideas] get() method for list and tuples

On Sun, 5 Mar 2017 at 14:08 Paul Moore wrote: > Self-evidently no. But what does that prove? That we should implement > list.get? You could use the dientical argument for *anything*. There > needs to be another reason for implementing it. > I don't think that's true. It's

Re: [Python-ideas] get() method for list and tuples

On Sun, Mar 5, 2017 at 10:13 AM, David Mertz wrote: > Specifically, if I think I'm dealing with a list that is likely to have 20 > items (rather than maybe 4 or fewer), I'm almost sure the best way to deal > with it is in a list (or comprehension, map(), etc) and NOT by poking

Re: [Python-ideas] get() method for list and tuples

On Sun, Mar 5, 2017 at 4:51 AM, Ed Kellett wrote: > It's hard to show examples because, generally speaking, when one can't do > a thing one does something else instead. I can restructure my programs to > avoid having this problem, or, if I'm in a hurry, I can use one of the

Re: [Python-ideas] Exploiting type-homogeneity in list.sort() (again!)

On Sat, Mar 4, 2017 at 11:19 PM, Elliot Gorokhovsky < elliot.gorokhov...@gmail.com> wrote: > (Summary of results: my patch at https://bugs.python.org/issue28685 makes > list.sort() 30-50% faster in common cases, and at most 1.5% slower in the > uncommon worst case.) > Thanks for the hard work,

Re: [Python-ideas] Add socket utilities for IPv4/6 dual-stack servers

Would it be possible to create a PyPI project to experiement the API and wait until we collected enough user feedback first? Currently socket is low level. Not sure if I like the idea of putting more high level features in it? Asyncio is a good place for high level features, but is limited to

Re: [Python-ideas] get() method for list and tuples

On 5 March 2017 at 13:03, Ed Kellett wrote: > > No. I'm asking: if list.get did exist, are there any cases (compatibility > with old versions aside) where list.get's semantics would be applicable, but > one of the alternatives would be the better choice? Self-evidently no. But

Re: [Python-ideas] get() method for list and tuples

On Sat, 4 Mar 2017 at 09:46 Steven D'Aprano wrote: On Fri, Mar 03, 2017 at 10:35:18PM +0100, Michel Desmoulin wrote: > Since the start of the discussion, contesters have been offering > numerous solutions, all being contextual and with gotchas, none being > obvious, simple

[Python-ideas] Add socket utilities for IPv4/6 dual-stack servers

Some years ago I started working on a patch for the socket module which added a couple of utility functions for being able to easily create a server socket, with the addition of being able to accept both IPv4 and IPv6 connections (as a single socket): https://bugs.python.org/issue17561 Given that