Re: [Python-ideas] Adding str.isascii() ?

On Fri, Jan 26, 2018 at 7:42 PM, INADA Naoki wrote: > Hi. > > Currently, int(), str.isdigit(), str.isalnum(), etc... accepts > non-ASCII strings. > s = １２３" s > '１２３' s.isdigit() > True print(ascii(s)) > '\uff11\uff12\uff13' int(s) > 123 > > But

Re: [Python-ideas] Adding str.isascii() ?

+1 The idea is not new and I like it. Naoki created https://bugs.python.org/issue32677 Victor 2018-01-26 11:22 GMT+01:00 Antoine Pitrou : > On Fri, 26 Jan 2018 17:42:31 +0900 > INADA Naoki > wrote: >> >> If str has str.isascii() method, it can be

Re: [Python-ideas] Adding str.isascii() ?

> No, because you can pass in maxchar to PyUnicode_New() and > the implementation will take this as hint to the max code point > used in the string. There is no check done whether maxchar > is indeed the minimum upper bound to the code point ordinals. API doc says: """ maxchar should be the true

Re: [Python-ideas] Adding str.isascii() ?

> +1 > > Just a note: checking the header in CPython will only give a hint, > since strings created using higher order kinds can still be 100% > ASCII. > Oh, really? I think checking header is enough for all ready unicode. For example, this is _PyUnicode_EqualToASCIIString implementation: if

Re: [Python-ideas] Adding str.isascii() ?

On 26.01.2018 09:53, Chris Angelico wrote: > On Fri, Jan 26, 2018 at 7:42 PM, INADA Naoki wrote: >> Hi. >> >> Currently, int(), str.isdigit(), str.isalnum(), etc... accepts >> non-ASCII strings. >> > s = １２３" > s >> '１２３' > s.isdigit() >> True >

[Python-ideas] Adding str.isascii() ?

Hi. Currently, int(), str.isdigit(), str.isalnum(), etc... accepts non-ASCII strings. >>> s = １２３" >>> s '１２３' >>> s.isdigit() True >>> print(ascii(s)) '\uff11\uff12\uff13' >>> int(s) 123 But sometimes, we want to accept only ascii string. For example, ipaddress module uses: _DECIMAL_DIGITS

Re: [Python-ideas] Adding str.isascii() ?

On 26.01.2018 10:44, INADA Naoki wrote: >> +1 >> >> Just a note: checking the header in CPython will only give a hint, >> since strings created using higher order kinds can still be 100% >> ASCII. >> > > Oh, really? > I think checking header is enough for all ready unicode. No, because you can

Re: [Python-ideas] Adding str.isascii() ?

On Fri, 26 Jan 2018 17:42:31 +0900 INADA Naoki wrote: > > If str has str.isascii() method, it can be simpler: > > `if s.isascii() and s.isdigit():` > > I want to add it in Python 3.7 if there are no opposite opinions. +1 from me. Regards Antoine.

Re: [Python-ideas] .then execution of actions following a future's completion

I have a simple way to solve this I believe. Why not just expose "_chain_future()" from asyncio/futures.py to the public, instead of copying and pasting parts of it? It already works, being used everywhere in the stdlib, it supports both asyncio and concurrent.futures, it's an easily testable

Re: [Python-ideas] Adding str.isascii() ?

2018-01-26 13:39 GMT+01:00 Steven D'Aprano : > I have no objection to isascii, but I don't think it goes far enough. > Sometimes I want to know whether a string is compatible with Latin-1 or > UCS-2 as well as ASCII. For that, I used a function that exposes the > size of code

Re: [Python-ideas] Adding str.isascii() ?

On Fri, Jan 26, 2018 at 10:17 PM, INADA Naoki wrote: >> No, because you can pass in maxchar to PyUnicode_New() and >> the implementation will take this as hint to the max code point >> used in the string. There is no check done whether maxchar >> is indeed the minimum

Re: [Python-ideas] Adding str.isascii() ?

2018-01-26 12:17 GMT+01:00 INADA Naoki : >> No, because you can pass in maxchar to PyUnicode_New() and >> the implementation will take this as hint to the max code point >> used in the string. There is no check done whether maxchar >> is indeed the minimum upper bound to

Re: [Python-ideas] Adding str.isascii() ?

2018-01-26 14:43 GMT+01:00 M.-A. Lemburg : > If that's indeed being used as assumption, the docs must be > fixed and PyUnicode_New() should verify this assumption as > well - not only in debug builds using C asserts() :-) As PyUnicode_FromStringAndSize(NULL, size),

Re: [Python-ideas] Adding str.isascii() ?

On Fri, Jan 26, 2018 at 05:42:31PM +0900, INADA Naoki wrote: > If str has str.isascii() method, it can be simpler: > > `if s.isascii() and s.isdigit():` > > I want to add it in Python 3.7 if there are no opposite opinions. I have no objection to isascii, but I don't think it goes far enough.

Re: [Python-ideas] Adding str.isascii() ?

Do you mean we should fix *all* of CPython unicode handling, not only str.isascii()? At least, equality test doesn't care wrong kind. https://github.com/python/cpython/blob/master/Objects/stringlib/eq.h

Re: [Python-ideas] Adding str.isascii() ?

On Fri, 26 Jan 2018 22:33:36 +0900 INADA Naoki wrote: > > > > Can you create a simple test-case that proves this? > > Sure. I think the question assumed "without writing custom C or ctypes code that deliberately builds a non-conformant unicode object" ;-) Regards

Re: [Python-ideas] Adding str.isascii() ?

On Fri, Jan 26, 2018, at 09:18, M.-A. Lemburg wrote: > Is there a way to call an API which fixes the setting > (a public version of unicode_adjust_maxchar()) ? > > Without this, how would an extension be able to provide a > correct value upfront without knowing the content ? It obviously has to

Re: [Python-ideas] Adding str.isascii() ?

On 26.01.2018 16:16, Random832 wrote: > On Fri, Jan 26, 2018, at 09:18, M.-A. Lemburg wrote: >> Is there a way to call an API which fixes the setting >> (a public version of unicode_adjust_maxchar()) ? >> >> Without this, how would an extension be able to provide a >> correct value upfront without

Re: [Python-ideas] Adding str.isascii() ?

On 26.01.2018 14:55, Victor Stinner wrote: > 2018-01-26 14:43 GMT+01:00 M.-A. Lemburg : >> If that's indeed being used as assumption, the docs must be >> fixed and PyUnicode_New() should verify this assumption as >> well - not only in debug builds using C asserts() :-) > > As

Re: [Python-ideas] Adding str.isascii() ?

On 26.01.2018 15:58, Antoine Pitrou wrote: > On Fri, 26 Jan 2018 22:33:36 +0900 > INADA Naoki > wrote: >>> >>> Can you create a simple test-case that proves this? >> >> Sure. > > I think the question assumed "without writing custom C or ctypes code > that deliberately

Re: [Python-ideas] Adding str.isascii() ?

No. See this mail. https://mail.python.org/pipermail/python-ideas/2018-January/048748.html Point is should we support invalid Unicode created by C API. And I assume no. 2018/01/26 午後11:58 "Antoine Pitrou" : On Fri, 26 Jan 2018 22:33:36 +0900 INADA Naoki

Re: [Python-ideas] Official site-packages/test directory

Guido van Rossum wrote: > IIUC another common layout is to have folders named test or tests inside > each package. This would avoid requiring any changes to the site-packages > layout. That's what I do for all my personal code. Yes, it means the test directories are shipped with the sdist, but

Re: [Python-ideas] Official site-packages/test directory

Hi Barry, Sometimes, I need to read the tests of a package because I don't understand the usage of a function/method/class and unfortunately, there is no documentation. In this case, and only in this case, I will try to find the tests and in the worst case, download the source and try to

Re: [Python-ideas] Adding str.isascii() ?

> > Can you create a simple test-case that proves this? Sure. $ git diff diff --git a/Modules/_testcapimodule.c b/Modules/_testcapimodule.c index 2ad4322eca..475d5219e1 100644 --- a/Modules/_testcapimodule.c +++ b/Modules/_testcapimodule.c @@ -5307,6 +5307,12 @@ PyInit__testcapi(void)

Re: [Python-ideas] Adding str.isascii() ?

On 26.01.2018 14:31, Victor Stinner wrote: > 2018-01-26 12:17 GMT+01:00 INADA Naoki : >>> No, because you can pass in maxchar to PyUnicode_New() and >>> the implementation will take this as hint to the max code point >>> used in the string. There is no check done whether

Re: [Python-ideas] .then execution of actions following a future's completion

On Fri, Jan 26, 2018 at 9:20 AM, Daniel Collins wrote: > @Guido As an aside, my understanding was libraries that fall back to c > (Numpy as an example) release the GIL for load heavy operations. But I > believe the explanation would hold in the general case if you replace

Re: [Python-ideas] .then execution of actions following a future's completion

@Guido As an aside, my understanding was libraries that fall back to c (Numpy as an example) release the GIL for load heavy operations. But I believe the explanation would hold in the general case if you replace thread with process using a ProcessPoolExecutor, that it would be good to be able

Re: [Python-ideas] Adding str.isascii() ?

We have _PyUnicodeWriter for such use cases. We may able to expose it as public API, but please start another thread for it. Unicode created by wrong maxchar is not supported from Python 3.3. == and hash() doesn't work properly for such unicode object. So str.isascii() has not to support it too.

Re: [Python-ideas] .then execution of actions following a future's completion

@Bar: I don't know about exposing _chain_future(). Certainly it's overkill for what the OP wants -- their PR only cares about chaining concurrent.future.Future. @Daniel: I present the following simpler solution -- it requires you to explicitly pass the executor, but since 'fn' is being submitted

Re: [Python-ideas] Adding str.isascii() ?

On Fri, Jan 26, 2018 at 12:42 AM, INADA Naoki wrote: > Currently, int(), str.isdigit(), str.isalnum(), etc... accepts > non-ASCII strings. > > >>> s = １２３" > >>> s > '１２３' > >>> s.isdigit() > True > >>> print(ascii(s)) > '\uff11\uff12\uff13' > >>> int(s) > 123 > > But

Re: [Python-ideas] .then execution of actions following a future's completion

So, just going point by point: Yes, absolutely put this off for 3.8. I didn’t know the freeze was so close or I would have put the 3.8 tag on originally. Yes, absolutely it is only meant for concurrent.futures futures, it only changes async where async uses concurrent.futures futures. Here’s

Re: [Python-ideas] .then execution of actions following a future's completion

Yeah, it would be better to use the chain_future call from async directly, but the problems are 1) it would make concurrent dependent on async and 2) if it were public, it would require users to instantiate futures, which they’re not supposed to do. -dancollins34 Sent from my iPhone > On Jan

Re: [Python-ideas] .then execution of actions following a future's completion

@Guido: I agree, that’s a much cleaner solution to pass the executor. However, I think the last line should be future.add_done_callback(callback) return newf not executor.submit. I’ll rewrite it like this and resubmit tonight for discussion. Sent from my iPhone > On Jan 26, 2018, at 11:59

Re: [Python-ideas] Dataclasses, keyword args, and inheritance

Even if we could inherit the setting, I would think that we would still want to require the code be explicit. It seems worse to implicitly require keyword only arguments for a class without giving any indication in the code. As it stands, the current implementation does not allow a later subclass

Re: [Python-ideas] .then execution of actions following a future's completion

That’s very true. I’ll try to keep my terminology more in line with the implementation in the future. The only problem with that, is that the function utilizes methods that are marked in the documentation as exclusively to be called by the executor (set_result, instantiation of future objects)

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

On 2018-01-26 08:18 PM, Chris Barker wrote: If there are robust and simple optimizations that can be added to CPython, great, but: This mail is the consequence of a true story, a story where CPython got defeated by Javascript, Java, C# and Go. at least those last three are

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

On Fri, Jan 26, 2018 at 02:18:53PM -0800, Chris Barker wrote: [...] > sure, but I would argue that you do need to write code in a clean style > appropriate for the language at hand. Indeed. If you write Java-esque code in Python with lots of deep chains obj.attr.spam.eggs.cheese.foo.bar.baz

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

On Sat, Jan 27, 2018 at 8:35 AM, Pau Freixes wrote: > def filter(rule, whatever): > if rule.x in whatever.x: > return True > > rules = get_rules() > whatevers = get_whatevers() > for rule in rules: > for whatever in whatevers: > if filter(rule,

[Python-ideas] Logging: a more perseverent version of the StreamHandler?

Hello, Some time ago, I set up some logging using stdout in a program with the `stdout_redirected()` context manager, which had to close and reopen stdout to work. Unsurprisingly, the StreamHandler didn't take it well. So I made a Handler class which is able to reload the stream (AKA get the

Re: [Python-ideas] .then execution of actions following a future's completion

Hm. Good point. (Though I'm not sure why that ban exists, since it's not enforceable.) Well, feel free to propose a new API for Python 3.8. On Fri, Jan 26, 2018 at 11:23 AM, Daniel Collins wrote: > That’s very true. I’ll try to keep my terminology more in line with the >

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

There are several reasons for the issues you are mentioning. 1. Attribute look up is much more complicated than you would think.  (If you have the time, watch https://www.youtube.com/watch?v=kZtC_4Ecq1Y that will explain things better than I can)  The series of operations that happen with every

[Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

Hi, This mail is the consequence of a true story, a story where CPython got defeated by Javascript, Java, C# and Go. One of the teams of the company where Im working had a kind of benchmark to compare the different languages on top of their respective "official" web servers such as Node.js,

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

On Sat, Jan 27, 2018 at 09:12:29AM +1100, Chris Angelico wrote: > Are you sure it's the language's fault? Failing to use a better data > type simply because some other language doesn't have it is a great way > to make a test that's "fair" in the same way that Balance and > Armageddon are "fair"

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

Hi, Well, I wrote https://faster-cpython.readthedocs.io/ website to answer to such question. See for example https://faster-cpython.readthedocs.io/mutable.html "Everything in Python is mutable". Victor 2018-01-26 22:35 GMT+01:00 Pau Freixes : > Hi, > > This mail is the

Re: [Python-ideas] Adding str.isascii() ?

IMO the special status for isascii() matches the special status of ASCII as encoding (yeah, I know, it's not the default encoding anywhere, but it still comes up regularly in standards and as common subset of other encodings). Should you wish to check for compatibility with other ranges IMO some

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

On Sat, Jan 27, 2018 at 10:07 AM, Steven D'Aprano wrote: > On Sat, Jan 27, 2018 at 09:12:29AM +1100, Chris Angelico wrote: > >> Are you sure it's the language's fault? Failing to use a better data >> type simply because some other language doesn't have it is a great way >> to

Re: [Python-ideas] Adding str.isascii() ?

On 1/27/2018 2:01 AM, Guido van Rossum wrote: On Fri, Jan 26, 2018 at 8:22 PM, Terry Reedy > wrote: On 1/26/2018 8:27 PM, Steven D'Aprano wrote: On Fri, Jan 26, 2018 at 02:37:14PM +0100, Victor Stinner wrote: Really? I

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

On 26Jan2018 20:59, Soni L. wrote: On 2018-01-26 08:18 PM, Chris Barker wrote: If there are robust and simple optimizations that can be added to CPython, great, but: This mail is the consequence of a true story, a story where CPython got defeated by Javascript,

Re: [Python-ideas] Adding str.isascii() ?

On Fri, Jan 26, 2018 at 02:37:14PM +0100, Victor Stinner wrote: > 2018-01-26 13:39 GMT+01:00 Steven D'Aprano : > > I have no objection to isascii, but I don't think it goes far enough. > > Sometimes I want to know whether a string is compatible with Latin-1 or > > UCS-2 as

Re: [Python-ideas] Adding str.isascii() ?

> > That's fine with me. Please also add it to bytes and bytearray objects. It's > okay if the implementation has to scan the string -- so do isdigit() etc. > > -- > --Guido van Rossum (python.org/~guido) Thanks for your pronouncement! I'll do it in this weekend. Regards, -- INADA Naoki

Re: [Python-ideas] Adding str.isascii() ?

On 1/26/2018 8:27 PM, Steven D'Aprano wrote: On Fri, Jan 26, 2018 at 02:37:14PM +0100, Victor Stinner wrote: Really? I never required such check in practice. Would you mind to elaborate your use case? tcl/tk and Javascript only support UCS-2 (16 bit) Unicode strings. Since IDLE is a

Re: [Python-ideas] Adding str.isascii() ?

On Fri, Jan 26, 2018 at 8:22 PM, Terry Reedy wrote: > On 1/26/2018 8:27 PM, Steven D'Aprano wrote: > >> On Fri, Jan 26, 2018 at 02:37:14PM +0100, Victor Stinner wrote: >> > > Really? I never required such check in practice. Would you mind to >>> elaborate your use case? >>> >>