[Python-ideas] Re: dunder methods for encoding & prettiness aware formal & informal representations

[Andrew Barnert via Python-ideas]

On Mar 20, 2020, at 19:32, Christopher Barker  wrote:
> 
> It’s a bit ironic: if you have a nifty idea for Python, you are often told to 
> try it out on your own. And if you expect it to maybe make its way into 
> Python, you’d want to use a dunder...
> 
> But then, dunders are reserved for the standard library. It’s a pickle.

When third-party libs end up in the stdlib, they usually change. Sometimes 
there are just small bikeshedding changes, like simplejson/json or 
statistics/stats, sometimes the whole thing gets redesigned, like 
attr.s/dataclasses or flufl.enum/enum. If you’re designing something that you 
hope will one day get standardized, maybe it’s not such a bad thing that you’re 
forced to think of how the migration is going to go for your early adopter 
users, and make them think about it too.

And if the stdlib version might change the semantics of the protocol in any 
way, which is more disruptive to users: the stdlib json module doesn’t respect 
your _tojson methods because it wants __json__, or the stdlib json module sort 
of respects your __json__ methods but gets it “wrong” in some important cases 
because it uses the same name but with changed semantics? And then, if you used 
_tojson, you can add the stdlib’s __json__ and the same module works as the 
backport of the 3.10 stdlib to older versions and as the more-featureful 
third-party module some people still need.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IT3SYBHNJCJXQUMAKGIQQ2OTFIRNMSSG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add the imath module

[Steven D'Aprano]

On Sat, Mar 21, 2020 at 04:18:32PM +, Jonathan Fine wrote:
> Hi
> 
> In https://bugs.python.org/issue40028 ,
> Ross Rhodes suggested adding to the standard library a function that finds
> the prime factors of a non-negative integer. So far as I know, any general
> algorithm for doing this requires a list of prime numbers.

No, that's incorrect. Having a precomputed list of prime numbers is 
impractical for heavy-duty factorization of even moderately largish 
numbers, let alone genuinely large numbers, although trial division by 
the first few primes is often useful to eliminate the easy cases.

But trying to hold onto a table of millions of primes is wasteful and 
most importantly unnecessary. Primes can be cheaply generated on the 
fly, as needed, and probably faster than they can be read off a hard 
drive:

https://primes.utm.edu/notes/faq/LongestList.html

There are 16,352,460,426,841,680,446,427,399 primes below `10**27`

https://oeis.org/A006880

so even if we used a table of compact 32-bit integers, rather than int 
objects, that would take over 50 thousand million petabytes of storage 
if my calculations are correct. There may be more compact ways to store 
it, but I doubt you could get it under a few million petabytes.

And 10**27 is not an especially large number. It has only 27 digits. 
`(2*17*31*101*157*199)**100` has over 900 digits, and sympy can 
factorise it effectively instantly, without needing to pre-compute a 
multiple petabyte table of primes :-)

(In fairness, I did kinda cheat by generating a number I knew was easily 
factorisable. Working on an arbitrary 900+ digit number isn't so fast.)


[...]
> > Consider
> > >>> 2**32
> > 4294967296

> > And if someone needs to find primes of this size, they've
> > probably got a spare gigabyte or two.

For what it's worth, I just found the above prime factorization on a 
computer with 145 MB available in my home directory. (I *really* need a 
new computer.)

In your research, you may consider 2**32 to be a large number, but to 
people working with primes (for fun, or for research), it's tiny. 
Thousands of digits is getting large; the largest known prime, as of 
January 2020, has over 24 million digits.


-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RJ5XESEZ7K6BUBE4WXBM5JTHWWIOSP2G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add the imath module

[Jonathan Fine]

Hi

In https://bugs.python.org/issue40028 ,
Ross Rhodes suggested adding to the standard library a function that finds
the prime factors of a non-negative integer. So far as I know, any general
algorithm for doing this requires a list of prime numbers.

To this issue I've just added
https://bugs.python.org/issue40028?#msg364757 which
reads:

A pre-computed table of primes might be better. Of course, how long should
> the table be. There's an infinity of primes.
>


> Consider
> >>> 2**32
> 4294967296
>


> This number is approximately 4 * (10**9). According to
> https://en.wikipedia.org/wiki/Prime_number_theorem, there are 50,847,534
> primes less than 10**9. So, very roughly, there are 200,000,000 primes less
> than 2**32.
>


> Thus, storing a list of all these prime numbers as 32 bit unsigned
> integers would occupy about
> >>> 200_000_000 / (1024**3) * 4
> 0.7450580596923828
> or in other words 3/4 gigabytes on disk.
>


> A binary search into this list, using as starting point the expected
> location provided by the prime number theorem, might very well require on
> average less than two block reads into the file that holds the prime number
> list on disk. And if someone needs to find primes of this size, they've
> probably got a spare gigabyte or two.
>


> I'm naturally inclined to this approach because by mathematical research
> involves spending gigahertz days computing tables. I then use the tables to
> examine hypotheses. See https://arxiv.org/abs/1011.4269. This involves
> subsets of the vertices of the 5-dimensional cube. There are of course
> 2**32 such subsets.


-- 
Jonathan
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SU5EIJ2O3OGPXHZMJ27G5JXSL2KMSAZ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dunder methods for encoding & prettiness aware formal & informal representations

[Stephen J. Turnbull]

Samuel Colvin writes:

 > > I'd be interested to see the "display utility" point of view expanded.
 > 
 > For me there are two use cases:

These both seem to be what I would consider "debugging" use cases, ie,
displaying whole objects to users who have some understanding of the
internals.  My understanding of "display utility" was for end users.
Perhaps Steven (d'Aprano) meant your "developers as users" case, but
that's not what I understood.

Also, am I to understand that simply adding "__pretty__" with a single
"debug" function gets all the benefits you describe, or do you need
additional support from devtool?

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VZ7FMSS5PD6ZOGPJ2G3A7NWDTLMDUHD5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add the imath module

[Ross]

Hello Serhiy,

I support this proposal, thanks for sharing this thread in my enhancement 
proposal (issue #40028). I would be more than happy to help you where time 
permits in implementing this module if approval is granted.

Ross
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FGED4WMHJBUIXX3OL2OLUEGKYHWWYSDH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dunder methods for encoding & prettiness aware formal & informal representations

[Samuel Colvin]

> I'd be interested to see the "display utility" point of view expanded.

For me there are two use cases:

I use devtool's debug() command instead of print() ALL THE TIME when
debugging, so does the rest of my office and others who I've demonstrated
its merits to. I use this sitecustomize trick
 to
avoid having to import debug().

Using debug() over print() or pprint() for debugging has the following
advantages, some more obvious than others:

   - It's much easier to read, particularly for big/complex/nested objects.
   Coloured output helps a lot here, so does better support for non std-lib
   objects like pandas dataframes, django querysets, multidict dictionaries,
   pydantic models etc.
   - I don't need to constantly write print('user.profile.age:',
   repr(user.profile.bio)) - just debug(user.profile.bio) takes care of
   showing the variable or expression I want to display as well as its value
   - debug() uses repr() which is mostly more useful when debugging than
   str()
   - debug() uses a slightly different logic to display nested objects
   compared to pprint(), you can think of it as looking much more like
   json.dumps(..., indent=4), maybe it's just that I'm the JSON generation,
   but I find it much easier to read than pprint(...)
   - having the file and line number saves me having to try and dig around
   to find where the print state is - this is particularly useful when
   debugging code in installed packages or in large code cases
   - debug() does a decent (if not perfect) job of displaying objects (e.g.
   strings) how I would want them in code so it can be super quick to add a
   test based on the value or copy it to code elsewhere
   - the sitecustomize trick means flake8 fails if I forget to remove
   debug() statements, similarly it means tests fail if I forget to remove them

The second case is displaying objects to developers-as-users. For
example aio-libs/aiohttp-devtools uses devtools to print request bodies and
headers to aid debugging of unexpected http response codes (as far as I
remember). Here I'm using devtools.pformat, it's convenient to use that
method rather than have to write lots of display logic in each package.

*Side Note:* I'm aware I'm talking about the devtools package (which I
maintain) a lot, I'm waiting for the moment someone get's cross that I'm
promoting my package. Sorry if it seems like that, but that's not what I'm
trying to do, devtools isn't perfect and I wish it didn't exist. I think
the python standard library should have a debug print command that knows
how to print common object types but also respects a dunder method on
objects which want to customise how they're displayed when debugging. The
name of the debug command and the name of the dunder method don't bother me
much. I'm just using devtools as an example of what I think should be in
the standard library.

Samuel
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AAG3FNAC5GMIJCG7DG5DKCYF2J5L2MRY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add the imath module

[Ross]

I support this proposal. This would fit nicely with my recent enhancement 
proposal raised to introduce a method for finding prime factors of non-negative 
integer n.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7DIOJ7XJZ3XQTC7S3J5TL5GCXU4SL7X4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dunder methods for encoding & prettiness aware formal & informal representations

[Stephen J. Turnbull]

Steven D'Aprano writes:

 > Oh, that's interesting. I mostly think of pretty-printing as a display 
 > utility aimed at end users.

Well, I'm mostly the only end-user of my code, so there's definitely a
bias here.  I'd be interested to see the "display utility" point of
view expanded.

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MNNOBGTR7MGM7ZPXJT2SGH7LW2NTKUQX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dunder methods for encoding & prettiness aware formal & informal representations

[Paul Moore]

On Sat, 21 Mar 2020 at 06:42, Steven D'Aprano  wrote:
>
> On Sat, Mar 21, 2020 at 12:16:29PM +0900, Stephen J. Turnbull wrote:
>
> > The way I think of pprint (FWIW, YMMV) is as a debug utility.
>
> Oh, that's interesting. I mostly think of pretty-printing as a display
> utility aimed at end users.

I agree with Stephen (ph) - debugging. For end user display, I'd
typically want to write a custom display function. For debugging, I
want a readable display with as little effort as possible (debugging
is hard enough without having to write display routines or put up with
unreadable data dumps).

Very much personal preference, though.

Paul
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5KH6HRU2TFRBWJNFL34GZ36DSJDC7TNB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New explicit methods to trim strings

[Rob Cliffe via Python-ideas]

Well, str.replace has a count parameter.  Presumably people use it (even 
if by accidentally discovering that without it, it replaces all 
occurrences when they only wanted one replaced).


On 18/03/2020 18:44, Alex Hall wrote:
Just the first occurrence. The vast majority of the time, that's what 
people want to do, and they will usually forget to add a 'count' 
parameter. Many people probably wouldn't even know it exists. It would 
be disastrous if code did the correct thing 99.9% of the time but 
occasionally silently mutilated a string.


On Wed, Mar 18, 2020 at 8:06 PM Rob Cliffe via Python-ideas 
mailto:python-ideas@python.org>> wrote:


Consider that the start or end of a string may contain repetitions
of an
affix.

Should `-+-+-+Spam'.stripprefix('-+')  remove just the first
occurence?
All of them?  Does it need a 'count' parameter?

[all modulo bikeshedding on the names of course]

Rob Cliffe

___
Python-ideas mailing list -- python-ideas@python.org

To unsubscribe send an email to python-ideas-le...@python.org

https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at

https://mail.python.org/archives/list/python-ideas@python.org/message/JMWBL7HILHKZ7JVN2JEH3K5NHQUVFVNZ/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/A4KY27KZK5UVVHRLBKUEYGOEA2NA7T3Y/
Code of Conduct: http://python.org/psf/codeofconduct/