[Python-ideas] Proposal: Complex comprehensions containing statements
This is a proposal for a new syntax where a comprehension is written as the
appropriate brackets containing a loop which can contain arbitrary statements.
Here are some simple examples. Instead of:
[
f(x)
for y in z
for x in y
if g(x)
]
one may write:
[
for y in z:
for x in y:
if g(x):
f(x)
]
Instead of:
lst = []
for x in y:
if cond(x):
break
z = f(x)
lst.append(z * 2)
one may write:
lst = [
for x in y:
if cond(x):
break
z = f(x)
yield z * 2
]
Instead of:
[
{k: v for k, v in foo}
for foo in bar
]
one may write:
[
for foo in bar:
{for k, v in foo: k: v}
]
## Specification
A list/set/dict comprehension or generator expression is written as the
appropriate brackets containing a `for` or `while` loop.
In the general case some expressions have `yield` in front and they become the
values of the comprehension, like a generator function.
If the comprehension contains exactly one expression statement at any level of
nesting, i.e. if there is only one place where a `yield` can be placed at the
start of a statement, then `yield` is not required and the expression is
implicitly yielded. In particular this means that any existing comprehension
translated into the new style doesn't require `yield`.
If the comprehension doesn't contain exactly one expression statement and
doesn't contain a `yield`, it's a SyntaxError.
### Dictionary comprehensions
For dictionary comprehensions, a `key: value` pair is allowed as its own
pseudo-statement or in a yield. It's not a real expression and cannot appear
inside other expressions.
This can potentially be confused with variable type annotations with no
assigned value, e.g. `x: int`. But we can essentially apply the same rule as
other comprehensions: either use `yield`, or only have one place where a
`yield` could be added in front of a statement. So if there is only one pair
`x: y` we try to implicitly yield that. The only way this could be
misinterpreted is if a user declared the type of exactly one expression and
completely forgot to give their comprehension elements, and the program would
almost certainly fail spectacularly.
### Whitespace
If placing the loop on a single line would be valid syntax outside a
comprehension (i.e. it just contains a simple statement) then we call this an
*inline* comprehension. It can be inserted in the same line(s) as other code
and formatted however the writer likes - there are no concerns about whitespace.
For a more complex comprehension, the loop must start and end with a newline,
i.e. the lines containing the loop cannot contain any tokens from outside,
including the enclosing brackets. For example, this is allowed:
foo = [
for x in y:
if x > 0:
f(x)
]
but this is not:
foo = [for x in y:
if x > 0:
f(x)]
This ensures that code is readable even at a quick glance. The eyes can quickly
find where the loop starts and distinguish the embedded statements from the
rest of the enclosing expression.
Furthermore, it's easy to copy paste entire lines to move them around, whereas
refactoring the invalid example above without specific tools would be annoying
and error-prone. It also makes it easy to adjust code outside the comprehension
(e.g. rename `foo` to something longer) without messing up indentation and
alignment.
Inside the loop, the rules for indentation and such are the same as anywhere
else. The syntax of the loop is valid only if it's also valid as a normal loop
outside any expression. The body of the loop must be more indented than the
for/while keyword that starts the loop.
### Variable scope
Since comprehensions look like normal loops they should maybe behave like them
again, including executing in the same scope and 'leaking' the iteration
variable(s). Assignments via the walrus operator already affect the outer
scope, only the iteration variable currently behaves differently. My
understanding is that this is influenced by the fact that there is little
reason to use the value of the iteration variable after a list comprehension
completes since it will always be the last value in the iterable. But since the
new syntax allows `break`, the value may become useful again.
I don't know what the right approach is here and I imagine it can generate
plenty of debate. Given that this whole proposal is already controversial and
likely to be rejected this may not be the best place to start discussion. But
maybe it is, I don't know.
## Benefits/comparison to current methods
### Uniform syntax
The new comprehensions just look like normal loops in brackets, or generator
functions. This should make them easier for beginners to learn than
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
On Thu, Feb 20, 2020 at 12:43 PM Steve Jorgensen wrote: > > > But frankly, it would be a rare case where this would be noticeable. > > -CHB > > Maybe uncommon, but I don't know about rare. Let's say you want to perform > list-wise computations, making new lists with results of operations on > existing lists (similar to numpy, but maybe trying to do something numpy is > unsuitable for)? You would want to pre-allocate the new array to the size > of the operand arrays. Not rate that you’d have a use case, but rate that the performance would be in issue. In past experiments, I’ve found the array re-allocation scheme is remarkably performant. On the other hand, all the methods suggested in this thread require at least a double allocation— which may not be noticeable in many applications, but it’s also a fairly light lift to make a single constructer for a pre-allocated array. And as Stephan pointed out — it would help in some high performance situations. One thing to keep In mind is that array.array is useful for use from C/Cython, when you don’t want the overhead of numpy. -CHB > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/2A2LOYFPBGGMOJGRLPTSLU6MSBWJPKV4/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WR6HJ2C6BQSB22KLHX5DHTBYC4AAG5XS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PYTHONLOGGING env variable
+1 On Thu, Feb 20, 2020 at 11:46 Alex Hall wrote: > Christopher Barker wrote: > > I think it’s a “Bad Idea” to use an environment variable — who knows what > > Python script may be running on a given system? > > But a standard command line argument to the interpreter could be useful. > > Can you clarify what the concern is about other Python scripts running? > Why doesn't that apply to [all the other special PYTHON* environment > variables]( > https://docs.python.org/3/using/cmdline.html#environment-variables), such > as `PYTHONWARNINGS` or `PYTHONVERBOSE`? > > Command line arguments only work if you're invoking the `python` command > directly. If you're running something above that like a bash script or a > package entrypoint it's more complicated. And configuring environment > variables is sometimes easier than changing a command or editing a script, > like when you're configuring a service hosted in the cloud (personally I'm > thinking of AWS ECS). > > I say do the same thing as so many other options and offer both a command > line argument and an environment variable. I don't think many people will > be setting the environment variable in their global shell profiles. When I > want to set PYTHONPATH I usually write: > > PYTHONPATH=/some/path python script.py > > which has no chance to affect anything else. > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/LXLSYULFPYNAVPFC3E66LLJQF2SCOER7/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido (mobile) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2FNOUYN25T22SALYB6VATFHJ3P6LMKDJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PYTHONLOGGING env variable
> Let's plan a cmdline argument then. > Should we go for -L much like -W? Yeah, I think a command-line argument is much more reasonable, less likely to cause issues with other implementations of logging configuration, and more practically useful. I'd be +1 on something like a `-L`. On Thu, Feb 20, 2020 at 2:05 PM Bar Harel wrote: > Let's plan a cmdline argument then. > > Should we go for -L much like -W? > Or -x logging for setting a default value of debug or info? > > Per python specification, -x is implementation dependent. > > I'll advocate for -L. I think the same concerns as -W are relevant for all > python implementations, and it gives more flexibility in terms of log level. > > Next question: Should -x dev include it? > If it was python day-1, I would say yes. My issue is the double inclusion > of stream handlers in case the logging was bootstrapped already. > So -x dev probably shouldn't include it. > > On Thu, Feb 20, 2020, 8:28 PM Christopher Barker > wrote: > >> I think it’s a “Bad Idea” to use an environment variable — who knows what >> Python script may be running on a given system? >> >> But a standard command line argument to the interpreter could be useful. >> >> -CHB >> >> On Thu, Feb 20, 2020 at 8:51 AM Mike Miller >> wrote: >> >>> >>> On 2020-02-19 17:12, Bar Harel wrote: >>> > Another idea I've had that may be of use: >>> > >>> > PYTHONLOGGING environment variable. >>> > >>> > Setting PYTHONLOGGING to any log level or level name will initialize >>> > logging.basicConfig() with that appropriate level. >>> > >>> > Another option would be that -x dev or a different -x logging will >>> initialize >>> > basic config. >>> >>> As a heavy user of logging, I like this one a lot. In a new project, >>> the lack >>> of such a feature allows one to stick with print() perhaps longer than >>> is prudent. >>> >>> -Mike >>> ___ >>> Python-ideas mailing list -- python-ideas@python.org >>> To unsubscribe send an email to python-ideas-le...@python.org >>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-ideas@python.org/message/US4RHU5S7MAEUJG7MPQIFZWM324IXNB6/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> -- >> Christopher Barker, PhD >> >> Python Language Consulting >> - Teaching >> - Scientific Software Development >> - Desktop GUI and Web Development >> - wxPython, numpy, scipy, Cython >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/YTYC4XPF5SWGWYI4DIGCXIEA53HMTNQU/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/KGRSK4U2L4MNMY57IG3XD52WHLOWG6OG/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XBLAIDGK36WBXBMZQGXIJ2WCS3VQ2DSE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
On Thu, Feb 20, 2020 at 2:11 PM Chris Angelico wrote:
> On Fri, Feb 21, 2020 at 8:52 AM Stephan Hoyer wrote:
> >
> > On Thu, Feb 20, 2020 at 12:41 PM Steve Jorgensen
> wrote:
> >>
> >> Christopher Barker wrote:
> >> ...
> >> > > Perhaps the OP wanted the internal array size initialized, but not
> used.
> >> > Currently the internal array will automatically be reallocated to
> grow as
> >> > needed. Which could be a performance hit if you know it’s going to
> grow
> >> > large.
> >> > But frankly, it would be a rare case where this would be noticeable.
> >> > -CHB
> >>
> >> Maybe uncommon, but I don't know about rare. Let's say you want to
> perform list-wise computations, making new lists with results of operations
> on existing lists (similar to numpy, but maybe trying to do something numpy
> is unsuitable for)? You would want to pre-allocate the new array to the
> size of the operand arrays.
> >
> >
> > Strong +1 for an array.zeros() constructor, and/or a lower level
> array.empty() which doesn't pre-fill values.
> >
>
> So it'd be a shorthand for something like this?
>
> >>> array.array("i", bytes(64))
> array('i', [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
>
> It'd be convenient to specify the size as a number of array elements
> rather than bytes. But I'm not a heavy user of array.array() so I
> won't say either way as to whether this is needed.
Yes, exactly.
The main problem with array.array("i", bytes(64)) is that memory gets
allocated twice, first to create the bytes() object and then to make the
array(). This makes it unsuitable for high performance applications.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/3PTSYLNOQ3I3FLNDS7MOJVCN6SFN2QGE/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
On Fri, Feb 21, 2020 at 8:52 AM Stephan Hoyer wrote:
>
> On Thu, Feb 20, 2020 at 12:41 PM Steve Jorgensen wrote:
>>
>> Christopher Barker wrote:
>> ...
>> > > Perhaps the OP wanted the internal array size initialized, but not used.
>> > Currently the internal array will automatically be reallocated to grow as
>> > needed. Which could be a performance hit if you know it’s going to grow
>> > large.
>> > But frankly, it would be a rare case where this would be noticeable.
>> > -CHB
>>
>> Maybe uncommon, but I don't know about rare. Let's say you want to perform
>> list-wise computations, making new lists with results of operations on
>> existing lists (similar to numpy, but maybe trying to do something numpy is
>> unsuitable for)? You would want to pre-allocate the new array to the size of
>> the operand arrays.
>
>
> Strong +1 for an array.zeros() constructor, and/or a lower level
> array.empty() which doesn't pre-fill values.
>
So it'd be a shorthand for something like this?
>>> array.array("i", bytes(64))
array('i', [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
It'd be convenient to specify the size as a number of array elements
rather than bytes. But I'm not a heavy user of array.array() so I
won't say either way as to whether this is needed.
ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/L3SZD4UIKS2BTBG5VLPV3QGI6PLW7EKI/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
On Thu, Feb 20, 2020 at 12:41 PM Steve Jorgensen wrote: > Christopher Barker wrote: > ... > > > Perhaps the OP wanted the internal array size initialized, but not > used. > > Currently the internal array will automatically be reallocated to grow as > > needed. Which could be a performance hit if you know it’s going to grow > > large. > > But frankly, it would be a rare case where this would be noticeable. > > -CHB > > Maybe uncommon, but I don't know about rare. Let's say you want to perform > list-wise computations, making new lists with results of operations on > existing lists (similar to numpy, but maybe trying to do something numpy is > unsuitable for)? You would want to pre-allocate the new array to the size > of the operand arrays. > Strong +1 for an array.zeros() constructor, and/or a lower level array.empty() which doesn't pre-fill values. A use case that came up for me recently is efficiently allocating and filling an object that satisfies the buffer protocol from C/C++ without requiring a NumPy dependency. As far as I can tell, there is no easy way to do this currently. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CJMX4EVEBLC23SVVVRSQHFZW7QYDBDRQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
Christopher Barker wrote: ... > > Perhaps the OP wanted the internal array size initialized, but not used. > Currently the internal array will automatically be reallocated to grow as > needed. Which could be a performance hit if you know it’s going to grow > large. > But frankly, it would be a rare case where this would be noticeable. > -CHB Maybe uncommon, but I don't know about rare. Let's say you want to perform list-wise computations, making new lists with results of operations on existing lists (similar to numpy, but maybe trying to do something numpy is unsuitable for)? You would want to pre-allocate the new array to the size of the operand arrays. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2A2LOYFPBGGMOJGRLPTSLU6MSBWJPKV4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add Binary module.
Guido van Rossum wrote: > On Thu, Feb 20, 2020 at 01:39 Steve Jorgensen ste...@stevej.name wrote: > > It seems to me that this could simply be a package on > > pypi rather than > > being added to the Python standard library. > > Sure. But the interesting part is how to design the API. I’ve seen a number > > of interesting ideas in this thread. > > --Guido (mobile) > > Yes. I certainly do agree with that. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/3XHHHL4XWTKJTKVWAA3RVSYT7PJPLUEP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PYTHONLOGGING env variable
Christopher Barker wrote: > I think it’s a “Bad Idea” to use an environment variable — who knows what > Python script may be running on a given system? > But a standard command line argument to the interpreter could be useful. Can you clarify what the concern is about other Python scripts running? Why doesn't that apply to [all the other special PYTHON* environment variables](https://docs.python.org/3/using/cmdline.html#environment-variables), such as `PYTHONWARNINGS` or `PYTHONVERBOSE`? Command line arguments only work if you're invoking the `python` command directly. If you're running something above that like a bash script or a package entrypoint it's more complicated. And configuring environment variables is sometimes easier than changing a command or editing a script, like when you're configuring a service hosted in the cloud (personally I'm thinking of AWS ECS). I say do the same thing as so many other options and offer both a command line argument and an environment variable. I don't think many people will be setting the environment variable in their global shell profiles. When I want to set PYTHONPATH I usually write: PYTHONPATH=/some/path python script.py which has no chance to affect anything else. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LXLSYULFPYNAVPFC3E66LLJQF2SCOER7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add Binary module.
On Thu, Feb 20, 2020 at 01:39 Steve Jorgensen wrote: > It seems to me that this could simply be a package on pypi rather than > being added to the Python standard library. > Sure. But the interesting part is how to design the API. I’ve seen a number of interesting ideas in this thread. -- --Guido (mobile) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZBXSD4RGITG2YWAE7MV7W6JKIVA5VRLG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PYTHONLOGGING env variable
I think it’s a “Bad Idea” to use an environment variable — who knows what Python script may be running on a given system? But a standard command line argument to the interpreter could be useful. -CHB On Thu, Feb 20, 2020 at 8:51 AM Mike Miller wrote: > > On 2020-02-19 17:12, Bar Harel wrote: > > Another idea I've had that may be of use: > > > > PYTHONLOGGING environment variable. > > > > Setting PYTHONLOGGING to any log level or level name will initialize > > logging.basicConfig() with that appropriate level. > > > > Another option would be that -x dev or a different -x logging will > initialize > > basic config. > > As a heavy user of logging, I like this one a lot. In a new project, the > lack > of such a feature allows one to stick with print() perhaps longer than is > prudent. > > -Mike > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/US4RHU5S7MAEUJG7MPQIFZWM324IXNB6/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YTYC4XPF5SWGWYI4DIGCXIEA53HMTNQU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PYTHONLOGGING env variable
On 2020-02-19 17:12, Bar Harel wrote: Another idea I've had that may be of use: PYTHONLOGGING environment variable. Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level. Another option would be that -x dev or a different -x logging will initialize basic config. As a heavy user of logging, I like this one a lot. In a new project, the lack of such a feature allows one to stick with print() perhaps longer than is prudent. -Mike ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/US4RHU5S7MAEUJG7MPQIFZWM324IXNB6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
On Thu, Feb 20, 2020 at 7:34 AM Serhiy Storchaka wrote: > 20.07.11 23:48, Sven Rahmann пише: > > What's missing is the possiblity to specify the final size of the > > array (number of items), especially for large arrays. > > array.array(typecode, [fillvalue]) * n Perhaps the OP wanted the internal array size initialized, but not used. Currently the internal array will automatically be reallocated to grow as needed. Which could be a performance hit if you know it’s going to grow large. But frankly, it would be a rare case where this would be noticeable. -CHB ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/56YLYLTWBOPEJ2GD4VFSTDZXR27Y6K4E/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/G4P7TWB4BZNOGZ6RDW2UJXSQHNZNCD36/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Pandas rookie
This last is for discussion of changes to the Python language itself, in
particular the CPython reference implementation. Python-list or a Pandas
forum are appropriate for this question.
That said, it sounds like you want df.value_counts(). But if not, follow-up
in a more relevant place.
On Thu, Feb 20, 2020, 7:39 AM James Lu wrote:
>
>
> On Wed, Feb 19, 2020 at 5:23 PM FilippoM
> wrote:
>
>>
>> Hi, I've got a Pandas data frame that looks like this
>>
>> In [69]: data.head
>> Out[69]:
>> > 0 AndroidVIDEO_OK
>> 1 Android 4.2.2 VIDEO_OK
>> 2 Android 9 VIDEO_OK
>> 3 iOS 13.3 VIDEO_OK
>> 4Windows 10 VIDEO_OK
>> 5 Android 9 VIDEO_OK
>> ......
>> 24 Windows 10 VIDEO_OK
>> 25Android 9 VIDEO_OK
>> 26Android 6.0.1 VIDEO_OK
>> 27 Windows XP VIDEO_OK
>> 28Android 8.0.0 VIDEO_FAILURE
>> 29 Android 6.0 VIDEO_OK
>> ......
>> 2994iOS 9.1 VIDEO_OK
>> 2995 Android 9 VIDEO_OK
>> 2996 Windows 10 VIDEO_OK
>> 2997 Android 9 VIDEO_OK
>> 2998 Windows 10 VIDEO_OK
>> 2999 iOS 13.3 VIDEO_OK
>>
>>
>> with 109 possible values of the OS columns and just two possible values
>> ()VIDEO_OK and VIDEO_FAILURE) in the status column.
>>
>> How can I use Pandas' dataframe magic to calculate, for each of the
>> possible 109 values, how many have VIDEO_OK, and how many have
>> VIDEO_FAILURE I have respectively?
>>
>> I would like to end up with something like
>>
>> In[]: num_of_oks{"iOS 13.3"}
>> Out: 15
>>
>> In[]: num_of_not_oks{"iOS 13.3"}
>> Out: 3
>>
>> I am trying to do some matplotlib scatter plotting
>>
>> Thanks
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>
> Have you considered using traditional unix tools, like cut and count? Or
> traditional SQL.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/B27PCY7Z7IPCGN4K5UCKOTISBG7DIJCW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/K6JT2JCHZTNXOLZNLWCZCDFBJY626YWE/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
20.07.11 23:48, Sven Rahmann пише: At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. array.array(typecode, [fillvalue]) * n ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/56YLYLTWBOPEJ2GD4VFSTDZXR27Y6K4E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Specify number of items to allocate for array.array() constructor
I discovered that same trick. It would be nice to have that specifically indicated in the documentation until/unless a length argument is added to the constructor. It would be nice for the supported operators to be documented at all, actually. I didn't realize that array.array had multiplication operator support at all until I got around to dir-ing an instance. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/P74LYDOFELXWCKLLI34YEZIZLKSLBF7A/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Pandas rookie
On Wed, Feb 19, 2020 at 5:23 PM FilippoM wrote:
>
> Hi, I've got a Pandas data frame that looks like this
>
> In [69]: data.head
> Out[69]:
> 0 AndroidVIDEO_OK
> 1 Android 4.2.2 VIDEO_OK
> 2 Android 9 VIDEO_OK
> 3 iOS 13.3 VIDEO_OK
> 4Windows 10 VIDEO_OK
> 5 Android 9 VIDEO_OK
> ......
> 24 Windows 10 VIDEO_OK
> 25Android 9 VIDEO_OK
> 26Android 6.0.1 VIDEO_OK
> 27 Windows XP VIDEO_OK
> 28Android 8.0.0 VIDEO_FAILURE
> 29 Android 6.0 VIDEO_OK
> ......
> 2994iOS 9.1 VIDEO_OK
> 2995 Android 9 VIDEO_OK
> 2996 Windows 10 VIDEO_OK
> 2997 Android 9 VIDEO_OK
> 2998 Windows 10 VIDEO_OK
> 2999 iOS 13.3 VIDEO_OK
>
>
> with 109 possible values of the OS columns and just two possible values
> ()VIDEO_OK and VIDEO_FAILURE) in the status column.
>
> How can I use Pandas' dataframe magic to calculate, for each of the
> possible 109 values, how many have VIDEO_OK, and how many have
> VIDEO_FAILURE I have respectively?
>
> I would like to end up with something like
>
> In[]: num_of_oks{"iOS 13.3"}
> Out: 15
>
> In[]: num_of_not_oks{"iOS 13.3"}
> Out: 3
>
> I am trying to do some matplotlib scatter plotting
>
> Thanks
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
Have you considered using traditional unix tools, like cut and count? Or
traditional SQL.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/B27PCY7Z7IPCGN4K5UCKOTISBG7DIJCW/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PYTHONLOGGING env variable
> > > If you want it only temporarily for debugging purposes, or permanently > for all scripts in your system, it's not really feasible. > > Hmm, I'm not sure that I like the idea of overriding the default logging > level being set system-wide, or that there would be an especially strong > use case for it. Instead, it seems more practically common to want to set > it on an individual run of a program temporarily. In such cases, using some > form of command-line parsing utility is a perfectly a viable option. > Well I meant more like a permanent "debug" mode, but the temporary case is probably the one more appealing to me and other users. > > We have debugging flags for warnings, faulthandlers, asyncio etc... > Arguably though, the easier it is to implement on your own, the less of a > need there is to be an environment variable to do it. Also, I think the > logging configuration should be on a per-program basis, rather than > globally (which is what env vars are typically used for). > I agree that logging configuration should be on a per program basis. Maybe a flag should be the solution, like '-Linfo'. When I said 'trivial', I didn't mean easy to implement. I meant that it's a much more common use case, and makes sense to have. -Wdefault is trivial to implement. Asyncio.debug is a single flag. Logging takes the same amount of lines, even more if you need to parse the args, but is much more common and needed. I've done a little stack overflow search, and there are tons of questions about having or creating such a feature manually. Last but not least: our suggested methodology for libraries is to use "NullHandler". If a library uses it, errors will not show up unless you set up a handler (with probably stdout for debug) on the root logger. If '-Linfo' or '-x logging' will be part of the possible debug cmdline options, these obscure bugs would be much easier to track. (I personally spent a few days to track one like that not long ago) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BETDBL6LWQ3IV6PHI4OL52PWWGRCNXAE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add Binary module.
It seems to me that this could simply be a package on pypi rather than being added to the Python standard library. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/C42EYAVOTKPW3W3SM3G6JGQQKIZK3FBJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Ability to specify item sizes for array.array instances in a platform-independent way
Currently, if I know that I want an `array.array` object with `itemsize` of 4
there is no way to do that without first determining what the item sizes are
for `'i'`/`'I'` and `'l'`/`'L'` on the current platform. Presumably, things
could get even more hairy with future platforms.
Below are some ideas for how to support explicit, platform-agnostic item size
specification.
Allow a non-str sequence with itemsize and signedness members to be given as
the `typecode` value.
`a = array.array((4, array.SIGNED))`
Allow a numeric `itemsize` value to be given as the first positional argument
instead of a `typecode` string, and have an optional named argument for
signedness, signed by default.
`a = array.array(4) # Signed by default.`
`a = array.array(4, signedness=array.UNSIGNED)`
Allow the "@" and "=" prefixes (same as in `struct` format strings) in
`typecode` strings. This is my least preferred because I won't always have the
typecode or prefix value choices memorized, and looking them up is an extra
step. Also, the appropriate size and signedness might be determined at runtime,
so having to write additional code to map from size/signedness to a typecode is
an unnecessary annoyance.
`a = array.array('=i') # Signed integer of "standard" integer size.`
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/Y5S75LPSREFKIXJWMB66FVWDCCC546PH/
Code of Conduct: http://python.org/psf/codeofconduct/
