[Python-ideas] Re: [dataclasses] add a NON_FIELDS sentinel after which all attributes are ignored.

2023-06-23 Thread Eric V. Smith via Python-ideas 

On 6/23/2023 11:34 AM, Joao S. O. Bueno wrote:



On Fri, Jun 23, 2023 at 12:18 PM Eric V. Smith  wrote:



On Jun 23, 2023, at 9:34 AM, Joao S. O. Bueno 
wrote:




On Fri, Jun 23, 2023 at 2:35 AM Jelle Zijlstra
 wrote:



El jue, 22 jun 2023 a las 8:22, Randolf Scholz
() escribió:

Dataclasses should provide a way to ignore a type hinted
attributes, and not consider them as fields.

For example, some attributes might be derived during
`__post_init__` from the values of the fields or other
variables.

If one wants to still type hint these attributes, one has
to awkward workarounds to avoid having dataclass
interpret them as fields.
(https://stackoverflow.com/questions/76532816)



But it’s not clear (to me) why not being a field is desirable. Why
is it important?


Can't know  - not my design, it was Randolf's question.
They might represent an internal state that should not be relayed on 
serialization or conversion, for example.
I can imagine some scenarios where I'd want some instance attributes 
to be shorter lived and non-transient,
although, I'd more likely build the class "manually" instead of a 
dataclass - or, more likely, put the dataclass under

a wrapper layer that would handle the "perishable" states.

I think you can make dataclasses itself ignore the field by judicious 
use of `field` parameters. If there's some code that's looking through 
`dataclasses.fields` but wants to ignore some fields, irrespective of 
what dataclasses is doing, then I'd say it's on the caller to have a 
list of fields to ignore.


But without knowing the use case, it's hard to say.

Eric

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PTJFEC3EQGP4TDDMF6TA7AENOBTOVMSO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [dataclasses] add a NON_FIELDS sentinel after which all attributes are ignored.

2023-06-23 Thread Eric V. Smith via Python-ideas 
On Jun 23, 2023, at 9:34 AM, Joao S. O. Bueno  wrote:On Fri, Jun 23, 2023 at 2:35 AM Jelle Zijlstra  wrote:El jue, 22 jun 2023 a las 8:22, Randolf Scholz () escribió:Dataclasses should provide a way to ignore a type hinted attributes, and not consider them as fields.

For example, some attributes might be derived during `__post_init__` from the values of the fields or other variables.

If one wants to still type hint these attributes, one has to awkward workarounds to avoid having dataclass interpret them as fields. (https://stackoverflow.com/questions/76532816)But it’s not clear (to me) why not being a field is desirable. Why is it important?Eric

I propose `NON_FIELDS` sentinel, analogous to `KW_ONLY`. (alternative name suggestions welcome). when writing a dataclass, all attributes after this sentinel are ignored and not considered fields.

```
@dataclass
class Foo:
    field0: int
    field1: int

    _: KW_ONLY

   fieldN: int

    _: NON_FIELDS

    attr0: int   # @dataclass will ignore this type hint.How is this different from `attr0: int = field(init=False)`?attr0 would be listed as a `field` in the introspectable attributes of the dataclass in this way.That is why I did not suggest that in my initial answer to Randolf on stackoverflow:https://stackoverflow.com/questions/76532816/type-hint-extra-attributes-not-fields/76533091#76533091I like the dataclasses.attribute idea, though - (but it will also require static type checking tools to review theirdataclass special casing - it looks like there is no escape from that).   
```

Additionally one could consider adding an `attribute` typing construct, such that `attr0: attribute[int]` would mark it as a non-field attribute.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/P67URFV2JJRFD6J5TXD44EEBO4IRTEYF/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HS5E5XNHKLO47Q6UPF5QVUCIK2FR6VSF/
Code of Conduct: http://python.org/psf/codeofconduct/

___Python-ideas mailing list -- python-ideas@python.orgTo unsubscribe send an email to python-ideas-le...@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/C5QJQT5YV7UOKFF57PWD4VSF4RWUDOSF/Code of Conduct: http://python.org/psf/codeofconduct/___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AIBVZMIW7KYLY3T5Y3FNK2POVSNBIUTJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Julia envy

2023-06-23 Thread Neal Becker 
I think we can do better if we use fromiter and supply the size as count:

np.fromiter ((u**2 for u in range(10)), dtype=float, count=10)

On Fri, Jun 23, 2023 at 10:39 AM Joao S. O. Bueno 
wrote:

> If you use Python's own arrays and generator expressions instead of list
> comprehension (by just dropping the `[ ]`s),
> you will get each number converted to the target type in memory as soon as
> it is calculated. (It will be a full
> Python float/int instance during the calculation itself, though).
>
> This won't work with the usual numpy array constructors as those need the
> array size beforehand - but if you know
> the size beforehand, there is probably a numpy constructor with no need to
> go through Python arrays first (but I don't know one by heart)
> ```
> import numpy as np
> import array
>
>  data = np.array(array.array("b", (int(127 * cos(i/100)) for i in
> range(628))), dtype="int8", copy=False)
> ```
>
>
> On Fri, Jun 23, 2023 at 10:53 AM Neal Becker  wrote:
>
>> One item I admire from Julia and miss in python/numpy,
>>
>> I often use the power of python list comprehension to process data.  This
>> data often needs to be converted to numpy for other operations, for
>> example
>> fancy indexing.  The fact that operations using comprehensions (which
>> produce lists) and operations on numpy arrays use different incompatible
>> data structures requires conversions between lists and numpy arrays.
>> Comprehensions in Julia produce arrays directly (I believe), removing the
>> need for conversions.
>>
>> I don't see any easy way to improve this.  Any ideas?
>>
>> Thanks,
>> Neal
>>
>> --
>> *Those who don't understand recursion are doomed to repeat it*
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/SRT377EAT4BAOFNMXXX7J7UFFQAJZBPZ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>

-- 
*Those who don't understand recursion are doomed to repeat it*
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5PUKWPVAAKOPD7BC4SGZHOCNYDAWY4PL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [dataclasses] add a NON_FIELDS sentinel after which all attributes are ignored.

2023-06-23 Thread Joao S. O. Bueno 
On Fri, Jun 23, 2023 at 12:18 PM Eric V. Smith  wrote:

> 
>
> On Jun 23, 2023, at 9:34 AM, Joao S. O. Bueno  wrote:
>
> 
>
>
> On Fri, Jun 23, 2023 at 2:35 AM Jelle Zijlstra 
> wrote:
>
>>
>>
>> El jue, 22 jun 2023 a las 8:22, Randolf Scholz ()
>> escribió:
>>
>>> Dataclasses should provide a way to ignore a type hinted attributes, and
>>> not consider them as fields.
>>>
>>> For example, some attributes might be derived during `__post_init__`
>>> from the values of the fields or other variables.
>>>
>>> If one wants to still type hint these attributes, one has to awkward
>>> workarounds to avoid having dataclass interpret them as fields. (
>>> https://stackoverflow.com/questions/76532816)
>>>
>>
> But it’s not clear (to me) why not being a field is desirable. Why is it
> important?
>

Can't know  - not my design, it was Randolf's question.
They might represent an internal state that should not be relayed on
serialization or conversion, for example.
I can imagine some scenarios where I'd want some instance attributes to be
shorter lived and non-transient,
although, I'd more likely build the class "manually" instead of a dataclass
- or, more likely, put the dataclass under
a wrapper layer that would handle the "perishable" states.



> Eric
>
>
>>> I propose `NON_FIELDS` sentinel, analogous to `KW_ONLY`. (alternative
>>> name suggestions welcome). when writing a dataclass, all attributes after
>>> this sentinel are ignored and not considered fields.
>>>
>>> ```
>>> @dataclass
>>> class Foo:
>>> field0: int
>>> field1: int
>>>
>>> _: KW_ONLY
>>>
>>>fieldN: int
>>>
>>> _: NON_FIELDS
>>>
>>> attr0: int   # @dataclass will ignore this type hint.
>>>
>>
>> How is this different from `attr0: int = field(init=False)`?
>>
>
> attr0 would be listed as a `field` in the introspectable attributes of the
> dataclass in this way.
> That is why I did not suggest that in my initial answer to Randolf on
> stackoverflow:
>
> https://stackoverflow.com/questions/76532816/type-hint-extra-attributes-not-fields/76533091#76533091
>
> I like the dataclasses.attribute idea, though - (but it will also require
> static type checking tools to review their
> dataclass special casing - it looks like there is no escape from that).
>
>
>
>>
>>
>>> ```
>>>
>>> Additionally one could consider adding an `attribute` typing construct,
>>> such that `attr0: attribute[int]` would mark it as a non-field attribute.
>>> ___
>>> Python-ideas mailing list -- python-ideas@python.org
>>> To unsubscribe send an email to python-ideas-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-ideas@python.org/message/P67URFV2JJRFD6J5TXD44EEBO4IRTEYF/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/HS5E5XNHKLO47Q6UPF5QVUCIK2FR6VSF/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/C5QJQT5YV7UOKFF57PWD4VSF4RWUDOSF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B3YHKCWMD2LN6VYNMGY4WHZWZFKGP3TC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Julia envy

2023-06-23 Thread Joao S. O. Bueno 
If you use Python's own arrays and generator expressions instead of list
comprehension (by just dropping the `[ ]`s),
you will get each number converted to the target type in memory as soon as
it is calculated. (It will be a full
Python float/int instance during the calculation itself, though).

This won't work with the usual numpy array constructors as those need the
array size beforehand - but if you know
the size beforehand, there is probably a numpy constructor with no need to
go through Python arrays first (but I don't know one by heart)
```
import numpy as np
import array

 data = np.array(array.array("b", (int(127 * cos(i/100)) for i in
range(628))), dtype="int8", copy=False)
```


On Fri, Jun 23, 2023 at 10:53 AM Neal Becker  wrote:

> One item I admire from Julia and miss in python/numpy,
>
> I often use the power of python list comprehension to process data.  This
> data often needs to be converted to numpy for other operations, for
> example
> fancy indexing.  The fact that operations using comprehensions (which
> produce lists) and operations on numpy arrays use different incompatible
> data structures requires conversions between lists and numpy arrays.
> Comprehensions in Julia produce arrays directly (I believe), removing the
> need for conversions.
>
> I don't see any easy way to improve this.  Any ideas?
>
> Thanks,
> Neal
>
> --
> *Those who don't understand recursion are doomed to repeat it*
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/SRT377EAT4BAOFNMXXX7J7UFFQAJZBPZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IJB2VJHRVY5TUXILQDV4CZJEKPPTWHP2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Julia envy

2023-06-23 Thread Neal Becker 
One item I admire from Julia and miss in python/numpy,

I often use the power of python list comprehension to process data.  This
data often needs to be converted to numpy for other operations, for example
fancy indexing.  The fact that operations using comprehensions (which
produce lists) and operations on numpy arrays use different incompatible
data structures requires conversions between lists and numpy arrays.
Comprehensions in Julia produce arrays directly (I believe), removing the
need for conversions.

I don't see any easy way to improve this.  Any ideas?

Thanks,
Neal

-- 
*Those who don't understand recursion are doomed to repeat it*
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SRT377EAT4BAOFNMXXX7J7UFFQAJZBPZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [dataclasses] add a NON_FIELDS sentinel after which all attributes are ignored.

2023-06-23 Thread Thiago Bellini Ribeiro 
A possible suggestion for your problem:

On strawberry-graphql we have a solution for that, by annotating the field
with "strawberry.Private"[1], like this:

  @strawberry.type
  class Foo:
  str_attr: str
  str_private_attr: Private[str]

That Private is defined as:

  Private = Annotated[T, StrawberryPrivate()]

Meaning we could also do "str_private_attr: Annotated[str,
StrawberryPrivate()]"

When introspecting the dataclass fields to generate the graphql schema, we
will check if the annotation is annotated, and if it is and contains a
"StrawberryPrivate()" instance in its __args__, we will exclude that field
from the graphql API.

[1]
https://github.com/strawberry-graphql/strawberry/blob/main/strawberry/private.py

On Fri, Jun 23, 2023 at 10:32 AM Joao S. O. Bueno  wrote:

>
>
> On Fri, Jun 23, 2023 at 2:35 AM Jelle Zijlstra 
> wrote:
>
>>
>>
>> El jue, 22 jun 2023 a las 8:22, Randolf Scholz ()
>> escribió:
>>
>>> Dataclasses should provide a way to ignore a type hinted attributes, and
>>> not consider them as fields.
>>>
>>> For example, some attributes might be derived during `__post_init__`
>>> from the values of the fields or other variables.
>>>
>>> If one wants to still type hint these attributes, one has to awkward
>>> workarounds to avoid having dataclass interpret them as fields. (
>>> https://stackoverflow.com/questions/76532816)
>>>
>>> I propose `NON_FIELDS` sentinel, analogous to `KW_ONLY`. (alternative
>>> name suggestions welcome). when writing a dataclass, all attributes after
>>> this sentinel are ignored and not considered fields.
>>>
>>> ```
>>> @dataclass
>>> class Foo:
>>> field0: int
>>> field1: int
>>>
>>> _: KW_ONLY
>>>
>>>fieldN: int
>>>
>>> _: NON_FIELDS
>>>
>>> attr0: int   # @dataclass will ignore this type hint.
>>>
>>
>> How is this different from `attr0: int = field(init=False)`?
>>
>
> attr0 would be listed as a `field` in the introspectable attributes of the
> dataclass in this way.
> That is why I did not suggest that in my initial answer to Randolf on
> stackoverflow:
>
> https://stackoverflow.com/questions/76532816/type-hint-extra-attributes-not-fields/76533091#76533091
>
> I like the dataclasses.attribute idea, though - (but it will also require
> static type checking tools to review their
> dataclass special casing - it looks like there is no escape from that).
>
>
>
>>
>>
>>> ```
>>>
>>> Additionally one could consider adding an `attribute` typing construct,
>>> such that `attr0: attribute[int]` would mark it as a non-field attribute.
>>> ___
>>> Python-ideas mailing list -- python-ideas@python.org
>>> To unsubscribe send an email to python-ideas-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/python-ideas@python.org/message/P67URFV2JJRFD6J5TXD44EEBO4IRTEYF/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/HS5E5XNHKLO47Q6UPF5QVUCIK2FR6VSF/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/C5QJQT5YV7UOKFF57PWD4VSF4RWUDOSF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Thiago Bellini Ribeiro | https://bellini.dev
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PYH5DM6EMZAIB3ZTLG5WLPYTGZTBTLQZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [dataclasses] add a NON_FIELDS sentinel after which all attributes are ignored.

2023-06-23 Thread Joao S. O. Bueno 
On Fri, Jun 23, 2023 at 2:35 AM Jelle Zijlstra 
wrote:

>
>
> El jue, 22 jun 2023 a las 8:22, Randolf Scholz ()
> escribió:
>
>> Dataclasses should provide a way to ignore a type hinted attributes, and
>> not consider them as fields.
>>
>> For example, some attributes might be derived during `__post_init__` from
>> the values of the fields or other variables.
>>
>> If one wants to still type hint these attributes, one has to awkward
>> workarounds to avoid having dataclass interpret them as fields. (
>> https://stackoverflow.com/questions/76532816)
>>
>> I propose `NON_FIELDS` sentinel, analogous to `KW_ONLY`. (alternative
>> name suggestions welcome). when writing a dataclass, all attributes after
>> this sentinel are ignored and not considered fields.
>>
>> ```
>> @dataclass
>> class Foo:
>> field0: int
>> field1: int
>>
>> _: KW_ONLY
>>
>>fieldN: int
>>
>> _: NON_FIELDS
>>
>> attr0: int   # @dataclass will ignore this type hint.
>>
>
> How is this different from `attr0: int = field(init=False)`?
>

attr0 would be listed as a `field` in the introspectable attributes of the
dataclass in this way.
That is why I did not suggest that in my initial answer to Randolf on
stackoverflow:
https://stackoverflow.com/questions/76532816/type-hint-extra-attributes-not-fields/76533091#76533091

I like the dataclasses.attribute idea, though - (but it will also require
static type checking tools to review their
dataclass special casing - it looks like there is no escape from that).



>
>
>> ```
>>
>> Additionally one could consider adding an `attribute` typing construct,
>> such that `attr0: attribute[int]` would mark it as a non-field attribute.
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/P67URFV2JJRFD6J5TXD44EEBO4IRTEYF/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HS5E5XNHKLO47Q6UPF5QVUCIK2FR6VSF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C5QJQT5YV7UOKFF57PWD4VSF4RWUDOSF/
Code of Conduct: http://python.org/psf/codeofconduct/