Re: [Python-ideas] Shuffled

[Arek Bulski]

> You've probably spent way more time debating shuffled here than you would
have needed to add a shuffled function to your test file.

Another presupposition. I did add it to my utility. I am proposing to add
this to the std library so that others dont have to.


pozdrawiam,
Arkadiusz Bulski
​​
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Steven D'Aprano]

This thread is just going around and around in circles, so I will make 
one final response and then I'm going to bow out.

On Thu, Sep 08, 2016 at 11:34:57AM +0200, Arek Bulski wrote:

> Needs to be shuffled().
> 
> https://github.com/construct/construct/blob/master/tests/test_all.py
> See? No way to put imperative code between tests.
>
> I need shuffled. Enough said.

Okay, okay. Suppose you convince us. You complain and stamp your feet 
and demand and insist and repeat "I need shuffled" ten thousand times 
until you wear down everyone and they give in. With the 3.6 feature 
freeze just a few days away, this new code is going into 3.7, which 
probably won't reach a stable version until March or April 2018.

So what happens between now and then?

I see that your package Construct supports Python versions 2.6, 2.7, 
3.3, 3.4 and 3.5. Are you ready to abandon ALL of those and support only 
3.7 and higher?

If not, then you can't even use the std lib implementation of shuffled() 
**because it won't exist**. At best you can test for a std lib 
implementation, falling back to your own when it doesn't exist.

So you have to write your own implementation regardless of what the std 
lib does. You cannot rely on the std lib version until you've dropped 
support for Python 3.6, which you haven't even got support for yet! If 
you support just *three* releases, the current release plus the two 
previous, you won't be able to rely on the std lib shuffled() until 
Python 3.9 comes out, which will probably be in 2021.

Maybe you do need shuffled(). But you can't use the one in the std lib 
until (probably) 2021 or later. What are you going to do for the next 
five years?

- Do without. Then you don't really need it.
- Write your own. Then you don't need it to be in the std lib.

Adding new functions to the std lib now is an investment for future 
code, not a necessity for current, existing code. If you need this 
shuffled function now, you know how to write it. Harsh, but true.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Paul Moore]

On 8 September 2016 at 10:34, Arek Bulski  wrote:
> That wont work because I would have to type the expression that is used as
> argument twice in a test. I need shuffled. Enough said.

You've probably spent way more time debating shuffled here than you
would have needed to add a shuffled function to your test file.

Your justification for the amount of time you've spent here, and that
you've persuaded others to spend (and are further asking to be spent
on maintenance) *has* to be more significant than "I needed shuffled()
for this one specific case". But it's hard to understand what the
*general* use cases you've (presumably) identified are - everyone
responding has been saying "but I don't personally have a use for
this". So in order to warrant the addition of a function to the
stdlib, you need to explain what benefits justify the cost - and you
simply haven't done that. The one case you've stated could have been
solved in 2 minutes by writing a one-off function. And no-one has said
"now that you mention it, I find the problem you had comes up a lot",
so we're still at the position where it's a specialised use.

The cost is non-trivial. Implementation is a relatively small cost,
design is slightly higher (see for example Nick's point that this
should be a method on the random instance as well as a module-level
function), documentation and tests need to be added, 3rd party
training material needs to be updated, bug reports need to be managed
and resolved, etc. The benefit? I'm not clear, beyond you not needing
to write a custom function in your test suite, and a couple of people
who weren't aware they could use PYTHONSTARTUP to add their own
personal functions being less frustrated by how random.shuffle
behaves. And some theoretical "people might find this useful" comments
that no-one has backed up with code samples.

Nick contributed some suggestions in terms of benefits around
teachability, and the improved design noted above. But the point here
is that you shouldn't expect others to justify your proposal for you -
you need to justify it yourself, or be prepared to accept "we don't
see the benefit" as a response.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Danilo J. S. Bellini]

>
> That wont work because I would have to type the expression that is used as
> argument twice in a test.

(lambda data: random.sample(data, len(data)))(container)

That lambda is actually your "shuffled"...

2016-09-08 6:34 GMT-03:00 Arek Bulski :

> > So, why can't you call random.shuffle(all_tests) if you want to run
> your tests in random order?
>
> I dont randomize test order. People should stop assuming that they know
> better. I need to randomize some arguments for one particular test and I
> cannot call shuffle between tests. Its a continous list of declarative
> tests. Needs to be shuffled().
>
> https://github.com/construct/construct/blob/master/tests/test_all.py
> See? No way to put imperative code between tests.
>
> > sample(container, len(container))
>
> That wont work because I would have to type the expression that is used as
> argument twice in a test. I need shuffled. Enough said.
>
> > I'll reiterate that I don't have a use case for this myself...
>
> I dont have a use case for half of what the std library offers. Or for
> type annotations. Asynchronous comprehesions, what is that? Do you see me
> rejecting those?
>
> > (sample having default of entire list size)
>
> That would work but would not be pretty. shuffled() is self explanatory
> and has a nice ring to it. Randomized list is not a sample by definition.
>
>
> pozdrawiam,
> Arkadiusz Bulski
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Danilo J. S. Bellini
---
"*It is not our business to set up prohibitions, but to arrive at
conventions.*" (R. Carnap)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Arek Bulski]

> So, why can't you call random.shuffle(all_tests) if you want to run your tests
in random order?

I dont randomize test order. People should stop assuming that they know
better. I need to randomize some arguments for one particular test and I
cannot call shuffle between tests. Its a continous list of declarative
tests. Needs to be shuffled().

https://github.com/construct/construct/blob/master/tests/test_all.py
See? No way to put imperative code between tests.

> sample(container, len(container))

That wont work because I would have to type the expression that is used as
argument twice in a test. I need shuffled. Enough said.

> I'll reiterate that I don't have a use case for this myself...

I dont have a use case for half of what the std library offers. Or for type
annotations. Asynchronous comprehesions, what is that? Do you see me
rejecting those?

> (sample having default of entire list size)

That would work but would not be pretty. shuffled() is self explanatory and
has a nice ring to it. Randomized list is not a sample by definition.


pozdrawiam,
Arkadiusz Bulski
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Nick Coghlan]

On 8 September 2016 at 15:05, Danilo J. S. Bellini
 wrote:
>> 1. The cognitive leap between shuffling and sampling isn't small
>
> I don't think so, actually the Fisher-Yates shuffle algorithm algorithm is
> an iterative sampling algorithm:
> https://gist.github.com/danilobellini/6384872

I'm not talking about mathematical equivalence, I'm talking about
making an unaided leap from "shuffle this deck of cards"
(random.shuffle) and "pick a card, any card" (random.choice) to
"choose a random sample from this population" (random.sample). I can
see people following that logic given suitable instruction (since they
really are closely related operations), but it's a tough connection to
see on your own.

>> 4. With a default, random.sample becomes more easily confused with
>> random.choice
>
> I don't think k=1 would be a good default sample size from a statistics
> point of view, but I get the point (I'm from a DSP background, where "a
> sample" means one single "value").

Likewise - it isn't that I think "1" would be a reasonable default,
it's that without the second argument being there, I lapse back into
DSP terminology rather than statistical terminology.

Requiring the second argument as random.sample() does today keeps
everything nicely unambiguous.

> Controling the random function is required for the function to be really
> pure, else its output won't depend only on the inputs (and there would be
> some "state" in that "implicit input"). That would also be a great feature
> when non-uniform (or external) random number generators are to be used. This
> seem to be something that only shuffle gives some control (among the
> functions we're talking about), or am I missing something?

The module level "functions" in random are just bound methods for a
default global random.Random() instance, so they're not truly pure -
there's interdependence there via the shared PRNG state.

However, by creating your *own* Random instance, or another object
that provides the same API, you can get a lot more control over
things, including reproducible behaviour for a given seed.

I'm not familiar with how shuffle works internally, but presumably
passing a non-uniform distribution is a way let you bias the shuffle
(the docs don't actually explain *why* you'd want to use a randomiser
other than the default).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Danilo J. S. Bellini]

>
> 1. The cognitive leap between shuffling and sampling isn't small
>
I don't think so, actually the Fisher-Yates shuffle algorithm algorithm is
an iterative sampling algorithm: https://gist.github.com/danilo
bellini/6384872


> 2. "shuffled" would be a more logical name for an out-of-place shuffle
> than "sample"
>
Agreed.


> 3. "Copy the whole container" would be a surprising default for a
> function called "sample"
>
Perhaps "sample the whole population" sounds strange. Indeed, statisticians
probably wouldn't be happy with that. That reminds me the variance
estimator with the extra unbiasing "-1" in the denominator versus the
population variance.


> 4. With a default, random.sample becomes more easily confused with
> random.choice
>
I don't think k=1 would be a good default sample size from a statistics
point of view, but I get the point (I'm from a DSP background, where "a
sample" means one single "value").

Controling the random function is required for the function to be really
pure, else its output won't depend only on the inputs (and there would be
some "state" in that "implicit input"). That would also be a great feature
when non-uniform (or external) random number generators are to be used.
This seem to be something that only shuffle gives some control (among the
functions we're talking about), or am I missing something?


2016-09-08 1:26 GMT-03:00 Nick Coghlan :

> On 8 September 2016 at 13:33, Danilo J. S. Bellini
>  wrote:
> > Nice to know about random.sample! =)
> >
> > I think what OP said can then be reduced to having the default k in
> > random.sample to be the iterable size. The existance of random.sample is
> a
> > very strong argument against "shuffled", and the only "feature" shuffled
> > would have that random.sample doesn't have is that default size.
>
> There are a few reasons I don't think defining a default for
> random.sample() would be a good answer to Arek's question:
>
> 1. The cognitive leap between shuffling and sampling isn't small
> 2. "shuffled" would be a more logical name for an out-of-place shuffle
> than "sample"
> 3. "Copy the whole container" would be a surprising default for a
> function called "sample"
> 4. With a default, random.sample becomes more easily confused with
> random.choice
>
> For the last two concerns, if I saw "result =
> random.sample(container)" and didn't already know about
> random.choice(), I'd expect it to behave like random.choice(). Even
> knowing they're different, I'd still need to do a double-take to make
> sure I was remembering which was which correctly. By contrast,
> "random.sample(container, 1)", "random.sample(container, k)",
> "random.sample(container, len(container))" are all clearly different
> from the single result primitive "random.choice(container)"
>
> One interesting (to me anyway) aspect of an out-of-place shuffle is
> that you can readily implement it based on *either* of the more
> primitive operations (in-place shuffle or random sample):
>
> def shuffled(container):
> result = list(container)
> random.shuffle(result)
> return result
>
> def shuffled(container):
> return random.sample(container, len(container))
>
> Writing down those two examples does highlight a potential refactoring
> benefit to having "out-of-place shuffle" as an explicitly represented
> concept in the core Random API: it can be defined in terms of shuffle
> or sample *on the same Random instance*, and hence automatically
> benefit when code switches from using the global random number
> generator to using a purpose-specific Random instance that allows
> greater control over the reproducibility of results (which can be very
> important for testing, games, modeling & simulation).
>
> The above helper functions are both flawed on that front: they
> hardcode the use of the default global random number generator. To
> permit use of a specific random instance, they need to be changed to:
>
> def shuffled(container, random=random):
> result = list(container)
> random.shuffle(result)
> return result
>
> def shuffled(container, random=random):
> return random.sample(container, len(container))
>
> and then used as "result = shuffled(original, my_random_instance)" if
> you decide to switch away from the global API.
>
> By contrast, a method based implementation could be refactored the
> exact same way as any other random.Random method:
>
> result = my_random_instance.shuffled(original)
>
> I'll reiterate that I don't have a use case for this myself, but I'll
> cite the key arguments I see in favour:
>
> - an out-of-place shuffle may be useful as a bridging concept between
> in place shuffling and out of place sampling (in either direction)
> - the presence of "shuffled" becomes a reminder that "shuffle" itself
> is an in-place operation
> - "write it yourself" isn't as simple as it first sounds due to the
> common migration away 

Re: [Python-ideas] Shuffled

[Nick Coghlan]

On 8 September 2016 at 13:33, Danilo J. S. Bellini
 wrote:
> Nice to know about random.sample! =)
>
> I think what OP said can then be reduced to having the default k in
> random.sample to be the iterable size. The existance of random.sample is a
> very strong argument against "shuffled", and the only "feature" shuffled
> would have that random.sample doesn't have is that default size.

There are a few reasons I don't think defining a default for
random.sample() would be a good answer to Arek's question:

1. The cognitive leap between shuffling and sampling isn't small
2. "shuffled" would be a more logical name for an out-of-place shuffle
than "sample"
3. "Copy the whole container" would be a surprising default for a
function called "sample"
4. With a default, random.sample becomes more easily confused with random.choice

For the last two concerns, if I saw "result =
random.sample(container)" and didn't already know about
random.choice(), I'd expect it to behave like random.choice(). Even
knowing they're different, I'd still need to do a double-take to make
sure I was remembering which was which correctly. By contrast,
"random.sample(container, 1)", "random.sample(container, k)",
"random.sample(container, len(container))" are all clearly different
from the single result primitive "random.choice(container)"

One interesting (to me anyway) aspect of an out-of-place shuffle is
that you can readily implement it based on *either* of the more
primitive operations (in-place shuffle or random sample):

def shuffled(container):
result = list(container)
random.shuffle(result)
return result

def shuffled(container):
return random.sample(container, len(container))

Writing down those two examples does highlight a potential refactoring
benefit to having "out-of-place shuffle" as an explicitly represented
concept in the core Random API: it can be defined in terms of shuffle
or sample *on the same Random instance*, and hence automatically
benefit when code switches from using the global random number
generator to using a purpose-specific Random instance that allows
greater control over the reproducibility of results (which can be very
important for testing, games, modeling & simulation).

The above helper functions are both flawed on that front: they
hardcode the use of the default global random number generator. To
permit use of a specific random instance, they need to be changed to:

def shuffled(container, random=random):
result = list(container)
random.shuffle(result)
return result

def shuffled(container, random=random):
return random.sample(container, len(container))

and then used as "result = shuffled(original, my_random_instance)" if
you decide to switch away from the global API.

By contrast, a method based implementation could be refactored the
exact same way as any other random.Random method:

result = my_random_instance.shuffled(original)

I'll reiterate that I don't have a use case for this myself, but I'll
cite the key arguments I see in favour:

- an out-of-place shuffle may be useful as a bridging concept between
in place shuffling and out of place sampling (in either direction)
- the presence of "shuffled" becomes a reminder that "shuffle" itself
is an in-place operation
- "write it yourself" isn't as simple as it first sounds due to the
common migration away from the random module functions to a custom
Random instance

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Steven D'Aprano]

On Wed, Sep 07, 2016 at 11:47:49PM -0300, Danilo J. S. Bellini wrote:
> Though I agree with the argument that inexperienced developers are
> [usually] worse, that's not the case here, unless anyone here is really
> trying to say the ones arguing for or against "shuffled" are inexperienced.
> These ad hominem won't bring us anywhere.

That is not what I said. Please be careful of accusing others of things 
they didn't say.

It was Sven who made the claim that inexperienced developers are better 
at seeing inconsistancies. I was responding specifically to that claim. 
I'm not saying anything about the experience, or lack of, of either Sven 
or Arek.


> No one seem to be arguing if "shuffled" is inconsistent, but the "shuffle"
> consistency is a subject that didn't reach consensus. And the point now
> seem to be whether "shuffled" is useful or not. If I understood correctly,
> that proposal is about writing code in a more functional style (or a
> expression-oriented programming style), with a pure function for shuffling.
> 
> I think there are other related subjects that can be said about
> sorted/shuffled. For example: why a list? Why not a generator? 

You cannot sort a sequence lazily. You have to sort the entire sequence 
before you can be sure which item comes first. Making sorted() an 
iterator by default doesn't give you any advantage: it still has to 
sort the entire sequence before it can yield the first item.


> Something
> like random.choice is useful to get one single value, but if one just want
> to see a few random values from a large input without repeating the values,
> what's the best approach?

random.sample()

 
> Does that mean you expect Python to provide appended() [...] extended()
> > [...]
> >
> Doesn't the "+" operator do that?

Of course it does. But the "consistency" argument would be "why is 
append a method not an operator? Why isn't + called appended?".

That's just spelling. It isn't that important.


> > [...] list.cleared()?
> 
> Huh? OP is clearly talking about avoiding state, avoiding "in-place"
> operations. A clear list would be an empty list [].

Of course. We can see that. But again, that's "inconsistent":

* the in-place operation is spelled `list.clear()`

* the functional operation is spelled `[]`

There is nothing wrong with that. The value of consistency is grossly 
exaggerated. Sometimes it really doesn't matter if similar operations 
are spelled differently.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Nick Coghlan]

On 8 September 2016 at 12:00, Steven D'Aprano  wrote:
> On Wed, Sep 07, 2016 at 11:43:59PM +0200, Sven R. Kunze wrote:
>
>> BUT experienced devs also need to recognize and respect the fact that
>> younger/unexperienced developers are just better in detecting
>> inconsistencies and bloody work-arounds.
>
> That is not a fact. It is the opposite of a fact -- inexperienced
> developers are WORSE at spotting inconsistencies, because they don't
> recognise deep consistencies.

There are also cases where we're internally inconsistent or lacking
internal integration because we're being consistent with different
external environments.

So we have both binary floating point and decimal floating point
because there are inherent conflicts in the underlying computational
models.

We have both the abc module (which deals strictly with runtime
classes) and the typing module (which deals strictly with type
categories) as those things are similar, but not the same.

We have both asyncio and traditional synchronous APIs as there are
essential conflicts between blocking and non-blocking models of
control flow.

Cross-platform secure network connectivity handling is a pain because
many of the underlying platform APIs are unreliably implemented and
poorly documented moving targets.

Cross-platform software distribution is a pain largely because major
platform vendors are typically competing to lock in developers, so
their incentives are aligned with making what we're trying to do
difficult rather than offering to help out.

For these kinds of cases, frustrations on the part of experienced
developers arise when folks ask entirely reasonable questions "Why is
 so complicated?", but then refuse to accept the answer of
"Because it's genuinely complex". Pretending complexity doesn't exist
doesn't help anyone - it means they're left to their own devices to
come up with answers, rather than receiving more explicit guidance at
the language level.

More directly relevant to the current discussion is the principle of
"Not every three line function needs to be a builtin (or even in the
standard library)". There are ways to collect objective data on the
relative usage of particular constructs (especially as hosting
platforms like GitHub team up with analytics platforms like Google Big
Query to make their data searchable, and initiatives like Software
Heritage aim to bring that kind of capability to all published open
source software), but one of the *simplest* criteria to apply, and the
one that experienced core developers can typically answer without
further research, is "Would the standard library benefit from this?".

When the answer to that is "No", then the default answer to requests
for API additions that are relatively easy for people to implement for
themselves is always going to be "No" - otherwise the stdlib will
collapse under its own weight (and the persistent "up and to the
right" trend on http://bugs.python.org/issue?@template=stats means
that there's a solid argument to be made that the standard library is
already bigger than we can effectively maintain at current maintenance
funding levels).

Cheers,
Nick.

P.S. I wrote 
http://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html
about this a few years ago, and I still think it's an accurate
description of the underlying causes of these conflicts

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Steven D'Aprano]

On Thu, Sep 08, 2016 at 12:22:28AM +0200, Sven R. Kunze wrote:

> About the confusion of returning None. It's not confusing in the sense 
> of "o my god, I did it wrong, I need to learn it", but more, like "I 
> used shuffle, BECAUSE it's the only one I could find in a hurry"

Then don't be in such a hurry.

Honestly, there is no substitute for professionalism. Outside of fantasy 
novels and movies, there has never been a "ticking time bomb" situation 
where a programmer has to write code to shuffle a list in six seconds or 
the world will be destroyed. There is *always* time to at least glance 
at the docs, and it is never more than a handful of keypresses away in 
the interactive interpreter:

help(random.shuffle)

And if you don't read the docs? Oh well, you find out soon enough once 
you actually test your code or have a code review. No harm done.

If the programmer doesn't do *any* of those things -- doesn't read the 
docs, doesn't test their code, doesn't do code reviews -- then there's 
no hope for them. We could provide shuffled() and shuffle() and they'll 
still end up picking the wrong one fifty percent of the time.

"My wild guess of how a function works was wrong, so now I'm cheesed off 
that the function doesn't do what I expect" is NOT a good reason to add 
things to the standard library.

> I would expect this from 
> Python as it already provides both alternatives for sorting".

Does that mean you expect Python to provide appended() as well as 
append(), extended() as well as extend(), popped() as well as pop(), 
cleared() as well as clear(), inserted() as well as insert(), removed() 
as well as remove()?

If your response is to say "Why do we need all those? Why on earth would 
anyone need to use list.cleared()?" then perhaps you can now understand 
why shuffled() has to prove itself too. Functions don't get added just 
because they *can* be added. They have to be worth it. Every function 
has costs as well as benefits. Each function adds:

- more code to download, compile, maintain
- more tests
- more documentation
- more for the user to learn
- more choices for the user to decide between
- more complexity
- more risk of bugs

Unless the proposal can demonstrate that the benefit outweighs the 
costs, it should not be added. Merely stating that you need it 
doesn't demonstate that a four-line helper function at the top of 
your module isn't sufficient.

For what's it is worth, if it were *my* call, I'd accept that the costs 
of adding this are low, but the benefits are just a *tiny* bit higher. 
But that's a judgement call, and if Raymond see the cost:benefit ratio 
going the other way, I have no objective argument to change his mind.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Steven D'Aprano]

On Wed, Sep 07, 2016 at 11:43:59PM +0200, Sven R. Kunze wrote:

> BUT experienced devs also need to recognize and respect the fact that 
> younger/unexperienced developers are just better in detecting 
> inconsistencies and bloody work-arounds.

That is not a fact. It is the opposite of a fact -- inexperienced 
developers are WORSE at spotting inconsistencies, because they don't 
recognise deep consistencies.

For example, the on-going controversy about mutable default arguments:

def func(x=0, y=[]):
x += 1
y.append(1)
print(x, y)

To the inexperienced Python developer this makes no sense. Why does 
Pythod reset the value of x by default, while y remembers the value it 
had from the previous call? That's an inconsistency.

But the more experienced dev realises that there is no inconsistency. 
Python behaves exactly the same way in both cases. Both x and y are 
treated the same: they both are treated as "Early Binding". The only 
difference is that 0 is immutable, and so x += 1 binds a new value to x, 
while [] is mutable, and y.append(1) modifies that value in place.

Try this instead:

def func(x=0, y=[]):
x += 1
y = y + [1]  # don't use +=
print(x, y)

The more experienced programmer might not *like* this behaviour, they 
may wish that Python used Late Binding instead, but it takes a very 
subtle and deep knowledge of the language to understand that this is not 
an inconsistency despite the superficial appearances.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Alexander Belopolsky]

On Wed, Sep 7, 2016 at 9:12 PM, Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:

> On Wed, Sep 7, 2016 at 8:14 PM, Arek Bulski  wrote:
> >
> > If you want to see the declarative tests, here it is.
> > https://github.com/construct/construct/blob/master/tests/test_all.py
>
>
> So, why can't you call random.shuffle(all_tests) if you want to run your
> tests in random order?


It may be instructive for you to see how this functionality is implemented
in CPython's own test suit:

https://github.com/python/cpython/blob/276f4ef97a434d4279a2d207daa34cafcf0994f7/Lib/test/libregrtest/main.py#L230
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Alexander Belopolsky]

On Wed, Sep 7, 2016 at 6:56 PM, Arek Bulski  wrote:

> In the project I maintain (construct) there are declarative testcases that
> look like a long list of (func, args, excepted output, error type) tuples.
> There is no way for me to call shuffle in there.


Can you explain why?  Something like this can be easily done with pytest:

In [1]: def foo(x):
   ...: return x + 1
   ...:

In [2]: import pytest

In [3]: @pytest.mark.parametrize('x, y', [
   ...: (100, 101),
   ...: (200, 201),
   ...: (300, 301),])
   ...: def test_foo(x, y):
   ...: assert foo(x) == y
   ...:

In [4]: test_foo.parametrize.args
Out[4]: ('x, y', [(100, 101), (200, 201), (300, 301)])

In [5]: import random

In [6]: random.shuffle(test_foo.parametrize.args[1])

In [7]: test_foo.parametrize.args
Out[7]: ('x, y', [(200, 201), (100, 101), (300, 301)])
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[David Mertz]

On Wed, Sep 7, 2016 at 3:10 PM, Sven R. Kunze  wrote:

> @David
> Your idea of a PyPI package could almost work. However, in an interactive
> python console, I expect as much batteries included as possible to make it
> as quick as possible. And considering how simple such wrapper would be, it
> almost does not warrant the download of a third-party package.
>

Of course I don't think it's worth having a PyPI package for this one
function.  That's why I suggested it could possibly live in `boltons` (that
already contains a lot of other simple functions that aren't in the stdlib)
or in `arek_utils` (which I haven't described except implicitly as a
collection of "a bunch of useful things").

Tim argues that this functionality is even too simple for inclusion in
boltons; that might be true.  But either way, it's not a huge burden in the
interactive console to type a first line of `from my_utils import *` before
getting down to work.



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Arek Bulski]

Thank you all for the meta discussion. I know we sidetracked a bit but
please everyone remember, this is an open discussion floor.

I can present you a use case, just didnt think I even needed one. This
seems so obvious to me that it should just go in without much saying. In
the project I maintain (construct) there are declarative testcases that
look like a long list of (func, args, excepted output, error type) tuples.
There is no way for me to call shuffle in there. I can only use shuffled.
So here it is, the use case you requested.

The argument that it can be easily implemented does not stick. In
interactive sessions, indeed writing own functions is a pain in the ass.
Expecting batteries included is rightful. Builtins like min() max() sum()
are sooo easy to implement on your own. So are some methods in itertools,
you can even find their sources in the docs. That doesnt mean batteries
should not be included.

shuffled = sorted(list, key=lambda x: randint)
This is wrong because sorting is comparison based and must be nlogn bound.
Shuffling is cheaper computationally. Also someone would have to come up
with a formal proof that it provides a uniform distribution which probably
was done for the shuffle implementaion.

Another argument still remains: there is a missing analogy between
sort-sorted and shuffle-shuffled.

I did NOT suggest making it a builtin. It should be added to random module.


>  "Also shuffle() should return self so mutating methods could be chained."

That was a side suggestion. I withdraw that part. Chaining methods is
sometimes useful but we can go without it.


pozdrawiam,
Arkadiusz Bulski
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Matt Gilson]

I lurk around here a lot more than I actually post -- But in this case I
find myself weighing in firmly on the side of "please don't add this".
It'd just add noise in the documentation and __builtins__ namespace.  There
are millions of useful functions that *could* be added to the `list` object
(or to the `__builtins__` namespace).  It's the job of the core dev team to
decide which are the most useful for everybody and this is one where (in my
opinion) the usefulness doesn't justify the additional clutter.

It seems like the primary argument for this is that some users would like
to have this functionality baked into an interactive session.  If that's
the case, then I'd like to point out that if you find yourself wanting this
in an interactive session frequently there is an interactive startup file

that you can modify so that this function will always be available to
*you* whenever
you start up an interactive session (and you can put whatever else in there
that you like as well :-).

I hope that helps.

On Wed, Sep 7, 2016 at 3:22 PM, Sven R. Kunze  wrote:

> Maybe, there's a misunderstanding here and I hope I didn't waste too much
> of your and my time.
>
> Not sure where I got this from but my last status was that Arek would like
> to have a "shuffled" function that does exactly what you described. Maybe,
> that doesn't fit the reports description but his pull request will tell the
> truth. :)
>
>
> About the confusion of returning None. It's not confusing in the sense of
> "o my god, I did it wrong, I need to learn it", but more, like "I used
> shuffle, BECAUSE it's the only one I could find in a hurry" and then
> ascertain "why the heck is there no alternative returning the shuffled
> result instead of overwriting my list? I would expect this from Python as
> it already provides both alternatives for sorting".
>
>
> Sven
>
>
>
> On 08.09.2016 00:02, Tim Peters wrote:
>
>> [Sven R. Kunze ]
>>
>>> I am not questioning experience which everyone in a team can benefit
>>> from.
>>>
>>>
>>> BUT experienced devs also need to recognize and respect the fact that
>>> younger/unexperienced developers are just better in detecting
>>> inconsistencies and bloody work-arounds. They simply haven't had to live
>>> with them for so long. Experienced devs just are stuck in a rut/are
>>> routine-blinded: "we've done that for years", "there's no better way".
>>>
>>>
>>> That's the way we do it in our teams. Employing the new guys as some
>>> sort of
>>> inconsistency detectors. This way, they learn to find their way around
>>> the
>>> code base and they can improve it by doing so.
>>>
>>> And I would never allow it in my team, to dismiss this kind of
>>> observation
>>> from new colleagues. It's invaluable as they will become routine-blinded
>>> as
>>> well.
>>>
>> I have been more than willing to discuss it, and I did not close the
>> issue report.  I did say I was opposed to it, but that's simply
>> because I am, and I explained there too _why_ I was opposed.
>>
>> Do you have anything to say about the specific proposal?  I doubt
>> either of us has found this meta-discussion useful.  I'm still looking
>> for a compelling use case.  The only concrete thing anyone has noted
>> in `shuffled()`'s favor so far is that sometimes they're surprised by
>> the behavior of random.shuffle(list) returning None in an interactive
>> shell (noted by you, and by another, and I cheerfully own up to being
>> a bit surprised by that too long ago).   But that's an observation
>> about `random.shuffle()`, not about the proposed `shuffled()`.
>>
>>
>> [...] I would be far more annoyed if, e.g.,

 random.shuffle(some_million_element_list)
>>>
>>   swamped my terminal with mountains of output.

>>> But you readily accept this behavior for "sorted"? That makes no sense at
>>> all.
>>>
>> Of course it does.  The only analogy to random.shuffle(big_list)
>> returning None that makes a lick of sense here is that big_list.sort()
>> also returns None.  IF a `shuffled()` function is introduced, then of
>> course it should return its result - just like `sorted()` returns its
>> result.
>>
>>
>> You can't both behaviors simultaneously, so the status quo wins.
 Indeed, the venerable status quo ;-)

>>> Nobody said to change "shuffle".
>>>
>> A verbatim quote from the first message in this thread:
>>
>>  "Also shuffle() should return self so mutating methods could be
>> chained."
>>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 

[image: pattern-sig.png]

Matt Gilson // SOFTWARE ENGINEER

E: m...@getpattern.com // P: 603.892.7736

We’re looking for beta testers.  Go here
 to sign up!

Re: [Python-ideas] Shuffled

[Sven R. Kunze]

On 07.09.2016 02:49, Chris Kaynor wrote:
I'll weigh in and say that I've had a few cases where I've wanted a 
shuffled function, but not many. The vast majority are interactive 
uses, where I want to get a sampling of data, and in those cases I'm 
normally just printing the output (often, by letting the REPL handle it).


I'm fairly sure I've never wanted a shuffled in the actual code, and 
shuffle is almost always what I want (or just pulling items at random).


Probably the most common case is to produce a list of random numbers 
in a (small) range. The current code looks roughly like:


import random
items = list(range(10))
random.shuffle(items)
items # this is interactive, so this prints it for me

As this does not come up often, I almost invariably write the 
following first:

import random
random.shuffle(range(10))

Then get no output and write the first form. It is not a major 
difference, and only comes up maybe a few times a year at most for me.




That sounds extremely familiar. I would say the interactive session is 
the clear use-case here. But don't ask me why I need a shuffled list of 
somethings (probably for some highly vicious and immoral reasons ;-) ).


@David
Your idea of a PyPI package could almost work. However, in an 
interactive python console, I expect as much batteries included as 
possible to make it as quick as possible. And considering how simple 
such wrapper would be, it almost does not warrant the download of a 
third-party package.


@Tim
Of course it's easy to write a wrapper function. But from what I gather 
here, this is not the point. In interactive sessions, I find it highly 
annoying when I need to define my own functions. Especially because I 
need to do it again in a new python session.


Somebody provided a one-line hack using "sorted" to emulate "shuffled". 
The statement basically is: shuffling is a special kind of sorting. So, 
I would expect the interface to work the same. That at least suffices 
for me to understand Arek's point of view.


I would even go so far as to say:

shuffled(my_list) # returns a new shuffled list
my_list.shuffle() # shuffles in-place

Allowing to plug-in the RNG, when needed.

Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Tim Peters]

[Sven R. Kunze ]
> I am not questioning experience which everyone in a team can benefit from.
>
>
> BUT experienced devs also need to recognize and respect the fact that
> younger/unexperienced developers are just better in detecting
> inconsistencies and bloody work-arounds. They simply haven't had to live
> with them for so long. Experienced devs just are stuck in a rut/are
> routine-blinded: "we've done that for years", "there's no better way".
>
>
> That's the way we do it in our teams. Employing the new guys as some sort of
> inconsistency detectors. This way, they learn to find their way around the
> code base and they can improve it by doing so.
>
> And I would never allow it in my team, to dismiss this kind of observation
> from new colleagues. It's invaluable as they will become routine-blinded as
> well.

I have been more than willing to discuss it, and I did not close the
issue report.  I did say I was opposed to it, but that's simply
because I am, and I explained there too _why_ I was opposed.

Do you have anything to say about the specific proposal?  I doubt
either of us has found this meta-discussion useful.  I'm still looking
for a compelling use case.  The only concrete thing anyone has noted
in `shuffled()`'s favor so far is that sometimes they're surprised by
the behavior of random.shuffle(list) returning None in an interactive
shell (noted by you, and by another, and I cheerfully own up to being
a bit surprised by that too long ago).   But that's an observation
about `random.shuffle()`, not about the proposed `shuffled()`.


>> [...] I would be far more annoyed if, e.g.,
>>
>> >>> random.shuffle(some_million_element_list)
>>
>>  swamped my terminal with mountains of output.

> But you readily accept this behavior for "sorted"? That makes no sense at
> all.

Of course it does.  The only analogy to random.shuffle(big_list)
returning None that makes a lick of sense here is that big_list.sort()
also returns None.  IF a `shuffled()` function is introduced, then of
course it should return its result - just like `sorted()` returns its
result.


>> You can't both behaviors simultaneously, so the status quo wins.
>> Indeed, the venerable status quo ;-)

> Nobody said to change "shuffle".

A verbatim quote from the first message in this thread:

"Also shuffle() should return self so mutating methods could be chained."
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Chris Kaynor]

On Tue, Sep 6, 2016 at 3:04 PM, Tim Peters  wrote:

> But would you _use_ it?  I'm still asking for use cases.
>
> When, e.g., I'm randomizing permutations for testing, the last thing I
> want is:
>
> while whatever:
> result = function_of_xs(shuffled(xs))
> check result and complain if it's wrong
>
> Why not?  Because no trace remains of _which_ permutation provoked the
> failure when a failure occurs.  Instead code looks like this:
>
> while whatever:
> shuffle(xs)
> result = function_of_xs(xs)  # or xs[:] if the function mutates
> its arg
> check result and complain that `xs` specifically provoked a failure
>
> Indeed, the only clear "use case" that makes sense I've been able to
> think of is:
>
> xs = shuffled(xs)
>
> But that's written more easily and efficiently today as
>
> shuffle(xs)
>
> This is in stark contrast to sorted(), where clear use cases abound.
> That didn't get in because it's hard to mimic (it's basically as easy
> as shuffled()), but because it's exactly what's wanted in all kinds of
> contexts in all kinds of code.
>

I'll weigh in and say that I've had a few cases where I've wanted a
shuffled function, but not many. The vast majority are interactive uses,
where I want to get a sampling of data, and in those cases I'm normally
just printing the output (often, by letting the REPL handle it).

I'm fairly sure I've never wanted a shuffled in the actual code, and
shuffle is almost always what I want (or just pulling items at random).

Probably the most common case is to produce a list of random numbers in a
(small) range. The current code looks roughly like:

import random
items = list(range(10))
random.shuffle(items)
items # this is interactive, so this prints it for me

As this does not come up often, I almost invariably write the following
first:
import random
random.shuffle(range(10))

Then get no output and write the first form. It is not a major difference,
and only comes up maybe a few times a year at most for me.

Chris
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Tim Peters]

[David Mertz ]
> This definitely feels like a case of "put it on PyPI."  Actually, maybe
> contribute to `boltons`, it feels like it might fit as a utility function
> there.

It's trivial to write such a function if it's truly needed - it would
be easier to write it from scratch than to remember which module it's
hiding in.  There's a "clever" 1-liner, but with no imagination at all
it's still dead obvious:

def shuffled(xs):
from random import shuffle
xs = xs[:]
shuffle(xs)
return xs

`boltons` doesn't typically bother with screamingly obvious things.


> While I wouldn't mind being able to type `from random import shuffled`, I
> don't have a bit problem instead typing `from boltons import shuffled`, nor
> even `from arek_utils import shuffled`.

But would you _use_ it?  I'm still asking for use cases.

When, e.g., I'm randomizing permutations for testing, the last thing I want is:

while whatever:
result = function_of_xs(shuffled(xs))
check result and complain if it's wrong

Why not?  Because no trace remains of _which_ permutation provoked the
failure when a failure occurs.  Instead code looks like this:

while whatever:
shuffle(xs)
result = function_of_xs(xs)  # or xs[:] if the function mutates its arg
check result and complain that `xs` specifically provoked a failure

Indeed, the only clear "use case" that makes sense I've been able to
think of is:

xs = shuffled(xs)

But that's written more easily and efficiently today as

shuffle(xs)

This is in stark contrast to sorted(), where clear use cases abound.
That didn't get in because it's hard to mimic (it's basically as easy
as shuffled()), but because it's exactly what's wanted in all kinds of
contexts in all kinds of code.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Tim Peters]

On Tue, Sep 6, 2016 at 2:48 PM, David Mertz  wrote:
> On Tue, Sep 6, 2016 at 1:15 PM, Sven R. Kunze  wrote:
>>
>> Their response: "Oh, I don't need it, let's close it."
>> Arek: "But I need it."
>
>
> This definitely feels like a case of "put it on PyPI."  Actually, maybe
> contribute to `boltons`, it feels like it might fit as a utility function
> there.
>
> While I wouldn't mind being able to type `from random import shuffled`, I
> don't have a bit problem instead typing `from boltons import shuffled`, nor
> even `from arek_utils import shuffled`.
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[David Mertz]

On Tue, Sep 6, 2016 at 1:15 PM, Sven R. Kunze  wrote:
>
> Their response: "Oh, I don't need it, let's close it."
> Arek: "But I need it."
>

This definitely feels like a case of "put it on PyPI."  Actually, maybe
contribute to `boltons`, it feels like it might fit as a utility function
there.

While I wouldn't mind being able to type `from random import shuffled`, I
don't have a bit problem instead typing `from boltons import shuffled`, nor
even `from arek_utils import shuffled`.


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Tim Peters]

[Sven R. Kunze ]
>>>
>>> ...
>>> He already convinced some people. Just not some venerable Python devs,
>>> which>> doesn't necessarily mean something at all.
>>>
>>> Their response: "Oh, I don't need it, let's close it."
>>> Arek: "But I need it."
>>>
>>> So, who's right now?

[Tim]
>> By default, the venerable Python devs ;-)

[Sven]
> Well, little bit arrogant? I hope technical arguments weigh more than status
> here. ;)

I haven't seen any technical arguments here.  You snipped the parts of
my reply where I emphasized the need for concrete use cases.  In the
absence of anything approaching that, it's a matter of taste, and then
- yes - decades of Python development experience do -  and should -
outweigh a newcomer's wish list.  Especially when, as appears to be
the case here, those with truly extensive experience agree.


> ...
> But I remember using it in interactive sessions (damn, why didn't I check
> that into git?). And there I remember finding it quite irritating not having
> a return value at all.
>
> >>> random.shuffle([1,2,3,4])
>
> Just gives me: nothing.
>
> Of course, the period of irritation lasted short and reading __doc__ helped
> a lot, but, well, I could have saved me some time. You get the idea.

No, I don't.  It's an obvious feature of Python's list object design
that mutating methods always return None.  Which means "obvious" after
someone has learned the language, not obvious the first time they try
it.  StackOverflow is equally full of newbie complaints that, e.g.,

some_list.sort().append(100)

gives an "incomprehensible"

AttributeError: 'NoneType' object has no attribute 'append'

error.  Once "ah, mutating methods generally return None" sinks in,
they never post any complaint like that again.

I would be far more annoyed if, e.g.,

>>> random.shuffle(some_million_element_list)

swamped my terminal with mountains of output.

You can't both behaviors simultaneously, so the status quo wins.
Indeed, the venerable status quo ;-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Sven R. Kunze]

On 06.09.2016 20:46, Bernardo Sulzbach wrote:

On 09/06/2016 03:37 PM, Sven R. Kunze wrote:


Besides being a silly argument, it's an interesting solution.

Does it really work? I remember Microsoft utilizing a similar approach
for their browser selection tool which led to a skewed probability
distribution. Maybe, I wrong here though.



Yes. The key is evaluated only once, so each element gets a 
pseudo-random number. Sorting this list leads to a shuffle.


Ah yes, that might make it work.

However, a super-linear shuffle, whilst Fisher-Yates is a linear 
solution and also requires less additional memory.


Lastly, although it is obvious, from a software engineering 
standpoint, this is a mere weird hack.


Sure, but it works. ;) Though I agree that's not really intuitive if you 
are looking for "just shuffle that list, please".



Cheers,
Sven

PS: here's the statistical analysis of the browser ballot issue with 
some explanations on the different algorithms on shuffling: 
http://www.robweir.com/blog/2010/02/microsoft-random-browser-ballot.html

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Sven R. Kunze]

On 06.09.2016 20:46, Tim Peters wrote:

[Sven R. Kunze ]

...
He already convinced some people. Just not some venerable Python devs, which
doesn't necessarily mean something at all.

Their response: "Oh, I don't need it, let's close it."
Arek: "But I need it."

So, who's right now?

By default, the venerable Python devs ;-)


Well, little bit arrogant? I hope technical arguments weigh more than 
status here. ;)



[I don't need it in my source code which is under version control.]


It's okay. You don't need to reiterate that you don't need it.

I also don't need it when looking at my checked-in source code (but it 
seems there are people who do).


But I remember using it in interactive sessions (damn, why didn't I 
check that into git?). And there I remember finding it quite irritating 
not having a return value at all.


>> random.shuffle([1,2,3,4])

Just gives me: nothing.
Of course, the period of irritation lasted short and reading __doc__ 
helped a lot, but, well, I could have saved me some time. You get the idea.



Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[אלעזר]

(Just to be clear, I wasn't trying to suggest this as more than an ad-hoc
solution for a throwaway script. But to me, "sorted by random key" is
almost as obvious as "shuffled", perhaps more so for non english speakers
with little background in CS terms; the words "sorted" and "random" jumps
to the eye, and sometimes you don't need more than that)
~Elazar

בתאריך יום ג׳, 6 בספט' 2016, 21:48, מאת Bernardo Sulzbach ‏<
mafagafogiga...@gmail.com>:

> On 09/06/2016 03:37 PM, Sven R. Kunze wrote:
> >
> > Besides being a silly argument, it's an interesting solution.
> >
> > Does it really work? I remember Microsoft utilizing a similar approach
> > for their browser selection tool which led to a skewed probability
> > distribution. Maybe, I wrong here though.
> >
>
> Yes. The key is evaluated only once, so each element gets a
> pseudo-random number. Sorting this list leads to a shuffle.
>
> However, a super-linear shuffle, whilst Fisher-Yates is a linear
> solution and also requires less additional memory.
>
> Lastly, although it is obvious, from a software engineering standpoint,
> this is a mere weird hack.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Bernardo Sulzbach]

On 09/06/2016 03:37 PM, Sven R. Kunze wrote:


Besides being a silly argument, it's an interesting solution.

Does it really work? I remember Microsoft utilizing a similar approach
for their browser selection tool which led to a skewed probability
distribution. Maybe, I wrong here though.



Yes. The key is evaluated only once, so each element gets a 
pseudo-random number. Sorting this list leads to a shuffle.


However, a super-linear shuffle, whilst Fisher-Yates is a linear 
solution and also requires less additional memory.


Lastly, although it is obvious, from a software engineering standpoint, 
this is a mere weird hack.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Tim Peters]

[Sven R. Kunze ]
> ...
> He already convinced some people. Just not some venerable Python devs, which
> doesn't necessarily mean something at all.
>
> Their response: "Oh, I don't need it, let's close it."
> Arek: "But I need it."
>
> So, who's right now?

By default, the venerable Python devs ;-)

As I said in my comments on the issue report, I searched through all
my code for instances of `shuffle()`, and didn't find any that could
be improved by `shuffled()`.  Nor could I dream up a compelling use
case.  So it wasn't just a knee-jerk "oh, I don't need it", the "I
don't need it" was the outcome of some research and thought.

In contrast, the original request was a bare "But I need it", without
so much as a single concrete use case to justify it ("testing" is an
application area, not a concrete use case - I've made extensive use of
shuffling in testing too, but - as I said - found no existing code
where `shuffled()` would have helped).

In the absence of compelling concrete use cases, new ideas have
approximately no chance of being adopted.  Otherwise the language &
libraries bloat in counterproductive ways.  Just ask any venerable
Python dev ;-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Sven R. Kunze]

Are you serious? The reason for "we don't need standardization" == 
"there is a solution, long, and with a lot of special characters". Holy 
c***. I remember Python being the least verbose language on the planet. 
I suspect this argument didn't lead to status quo. It's like saying, we 
don't need variable annotations because we have comments.



On 06.09.2016 20:25, אלעזר wrote:

Naive shuffled() can be emulated using a single expression:

   sorted(lst, key=lambda _: random())

So there's even less incentive for standardization.


Besides being a silly argument, it's an interesting solution.

Does it really work? I remember Microsoft utilizing a similar approach 
for their browser selection tool which led to a skewed probability 
distribution. Maybe, I wrong here though.



Cheer,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[אלעזר]

Naive shuffled() can be emulated using a single expression:

   sorted(lst, key=lambda _: random())

So there's even less incentive for standardization.

~Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Chris Angelico]

On Wed, Sep 7, 2016 at 4:15 AM, Sven R. Kunze  wrote:
> It's community project after all.

What's that mean, exactly? That the community gets to vote on what
goes into Python? Because we don't.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Sven R. Kunze]

Hi Steven,

On 06.09.2016 18:32, Steven D'Aprano wrote:

On Tue, Sep 06, 2016 at 05:29:18PM +0200, Arek Bulski wrote:

[I think Arek is quoting me here]

One moderately stong piece of evidence would be if this function is widely
available in third-party libraries and other languages.

Wrong. Python is NOT known for doing everything by how it was done before.

That's not what I said.

I'm telling you what you need to do to have the best chance of your
proposal being accepted. It isn't enough to just write a patch.

Python is not *your* language where you get to decide what goes in and
out, it is Guido's language, and he trusts the community to come to a
consensus for these things. Part of that process is to trust respected
core developers like Raymond Hettinger and Tim Peters to make decisions.

I have been following the Python-Ideas mailing list for, oh, probably
ten years. Maybe fifteen. I've seen hundreds, maybe thousands of
suggestions and ideas rejected, and dozens accepted, including some
of my own.

If your only argument is to continue to insist that Python should have a
shuffled() function and you'll write a patch, it will go no where. First
you have to convince people that the patch is needed.


He already convinced some people. Just not some venerable Python devs, 
which doesn't necessarily mean something at all.


Their response: "Oh, I don't need it, let's close it."
Arek: "But I need it."

So, who's right now?

I would suggest not being so protective. It's community project after 
all. Maybe, the patch will help him understand that it's not so easy, or 
other people will see the benefit after reading it. If the patch isn't 
worth including, he learned something in the process of creating it.



My father used statically typed variables and his father used statically
typed variables so I will use...

And my grandfather and my father used dynamically typed variables. Lisp
had dynamically typed variables back in the 1950s, and Smalltalk had
them in the 1980s. So what?





Best,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Steven D'Aprano]

On Tue, Sep 06, 2016 at 05:29:18PM +0200, Arek Bulski wrote:

[I think Arek is quoting me here]
> > One moderately stong piece of evidence would be if this function is widely
> > available in third-party libraries and other languages.
> 
> Wrong. Python is NOT known for doing everything by how it was done before.

That's not what I said.

I'm telling you what you need to do to have the best chance of your 
proposal being accepted. It isn't enough to just write a patch.

Python is not *your* language where you get to decide what goes in and 
out, it is Guido's language, and he trusts the community to come to a 
consensus for these things. Part of that process is to trust respected 
core developers like Raymond Hettinger and Tim Peters to make decisions.

I have been following the Python-Ideas mailing list for, oh, probably 
ten years. Maybe fifteen. I've seen hundreds, maybe thousands of 
suggestions and ideas rejected, and dozens accepted, including some 
of my own.

If your only argument is to continue to insist that Python should have a 
shuffled() function and you'll write a patch, it will go no where. First 
you have to convince people that the patch is needed.


> My father used statically typed variables and his father used statically
> typed variables so I will use...

And my grandfather and my father used dynamically typed variables. Lisp 
had dynamically typed variables back in the 1950s, and Smalltalk had 
them in the 1980s. So what?


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] shuffled

[Arek Bulski]

They closed it. Will they accept a patch if I send it?

Where is the repo for the code? Never submitted anything to py code base.

-- Arkadiusz Bulski --
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Mahmoud Hashemi]

I tend to agree with Arek. I've been bitten multiple times, including once
yesterday, because shuffle works in place, when I really expect a
sorted()-like behavior for a standalone function like that.

Mahmoud
https://github.com/mahmoud
http://sedimental.org

On Mon, Sep 5, 2016 at 6:59 PM, Arek Bulski  wrote:

> shuffled() should be in the random module, of course. I dont suggest a
> builtin. Although now that you mentioned it, I could go for that too.
>
> There are usage cases where its heavily used, in randomized testing for
> example. I am sure that there are also other domains where randomization of
> lists is used.
>
> Another reason to put it there is that using shuffle is inconvenient. The
> fact that I CAN write it myself doesnt mean that it doesnt belong in the
> standard library.
>
> Implementing this in pure python wont take a lot of work.
>
> pozdrawiam,
> Arkadiusz Bulski
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Steven D'Aprano]

On Tue, Sep 06, 2016 at 03:59:10AM +0200, Arek Bulski wrote:

> Implementing this in pure python wont take a lot of work.

Great. Well, here's a tracker issue for it. 

http://bugs.python.org/issue27964

Can you provide a patch? Don't forget the tests and documentation.

Providing a patch doesn't mean the patch will be accepted. But no patch 
means that, even if accepted, you are unlikely to see anything before 
Python 3.7, as the 3.6 feature freeze is less than a week away.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Shuffled

[Arek Bulski]

shuffled() should be in the random module, of course. I dont suggest a
builtin. Although now that you mentioned it, I could go for that too.

There are usage cases where its heavily used, in randomized testing for
example. I am sure that there are also other domains where randomization of
lists is used.

Another reason to put it there is that using shuffle is inconvenient. The
fact that I CAN write it myself doesnt mean that it doesnt belong in the
standard library.

Implementing this in pure python wont take a lot of work.

pozdrawiam,
Arkadiusz Bulski
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Shuffled

[Arek Bulski]

We should add shuffled() analog to sorted. Also shuffle() should return
self so mutating methods could be chained.

-- Arkadiusz Bulski --
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/