[Python-Dev] Re: Declarative imports
On Sun, 10 Apr 2022, 15:53 Guido van Rossum, wrote: > On Sun, Apr 10, 2022 at 2:31 AM Daniel Pope wrote: > >> On Fri, 8 Apr 2022, 17:44 Guido van Rossum, wrote: >> >>> The interesting idea here seems to make "lazy imports" easier to >>> implement by making them explicit in the code. So far, most lazy import >>> frameworks for Python have done hacks with `__getattribute__` overrides. >>> >> >> The value is more than ease of implementation. Having syntax for import >> expressions makes them statically analysable, which is needed for type >> checkers and IDE autocompletion. >> > > This has been brought up a few times and I don't get it. Currently a use > of an imported module is perfectly analyzable by all the static type > checkers I know of (e.g. mypy, Pyre, pyright). > I was comparing a hypothetical import expression syntax with alternatives like __getattribute__ hacks, which I take to mean somemagicobject.package.module.attribute, or as other have suggested, importlib.import_module(). I believe those are not statically analysable at least without special casing them in the type checker. An import expression would be just as statically analysable as a statement, while being more ergonomic in certain situations. > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2UXMBKQJHN5AM3FKDUZ4DEF7TPD4HGYP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022, 17:44 Guido van Rossum, wrote: > The interesting idea here seems to make "lazy imports" easier to implement > by making them explicit in the code. So far, most lazy import frameworks > for Python have done hacks with `__getattribute__` overrides. > The value is more than ease of implementation. Having syntax for import expressions makes them statically analysable, which is needed for type checkers and IDE autocompletion. But I also see value in being able to type out code that uses modules not yet imported without breaking my flow to add an import statement. I don't yet trust IDEs to do this because I've been bitten by them doing so incorrectly in the past. The key questions to me are > - What should the notation be? > I would like to bid again for (import package.module) as an expression. Instead of doing the import and assigning package to a variable package it would evaluate to the module object package.module. The `as` form is not needed because no name is assigned and the `from` form isn't as valuable because you can just use attribute access afterwards. It isn't terse but it does make use of the import keyword and is thus instantly recognisable. It is even syntax highlighted correctly by much existing software. If we're using the import keyword then I think it has to look like this. But I concede that it isn't particularly elegant to type hint things with (import collections.abc).Mapping ...but not so inelegant that I couldn't see myself using it for a few one-off imports per module. A quirk is that it means there's a big difference between the statements import foo and (import foo) because one assigns a variable. I don't mind that; I don't think it is too ambiguous to a reader. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/T6V7WZAFEXGWMHXLHS7XHYXI5OPMOZKA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 12:23, Steve Dower wrote: > > I've read the rest of the thread so far, and agree strongly that we > can't do this at the language/runtime level. You mean the hoisting, right? I don't see any reason why an import expression without hoisting would be impractical. But I'd like to hear your thoughts if you think it is. Desirability on the other hand is subjective. I think I actually do desire it, others are not bothered. I don't see strong arguments as to why you definitely wouldn't want it in your language. Design is hard, but designing this is definitely not as hard as designing match/case or except* statements. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/G4TOKDQPJPKNEGPUGYMJFOHE7FSID2OY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 11:26, Malthe wrote: > Perhaps `some_regex = re::compile(r"...")` could work. > > That is, :: to delineate the import. As I mentioned in the Discourse thread, this syntax chimes with Rust (and C++, and also PHP, triggering fond memories of Paamayim Nekudotayim). There are two problems, sort of on the minor side: First, there's one place in Python where this can occur and that is in slices: xs[start::step] means the slice from start to the end with a step of step. But xs[(module::name)] is currently a syntax error, so you could just require parentheses to disambiguate the case. Still, it is confusing if you write something slightly more complex like xs[1 + math::factorial(n)] and you get NameError: name 'math' is not defined. Secondly, it precludes importing just a module object, you have to import a name from a module. Maybe using a module object in an expression is niche and can be disregarded. Or a solution could be to allow an expression like (re::) to import a module object, which I guess we could get used to. Also fun would be to consider what ::name does - access to current globals without the global keyword? I think this discussion is better suited to the python-ideas mailing list, or the Discourse thread I linked earlier. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MO524TQHUR7PMUWOZ4PBQAXDMOOF2KFP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022, 09:30 Malthe, wrote: > For example, `some_regex = @re.compile(...)`. > I like the idea of import expressions. I pitched it on Discourse recently: https://discuss.python.org/t/import-expressions/11582 However, I do not see hoisting as something that should be done by Python. In that thread it was suggested your IDE could do it, or maybe isort, which seems fine because it's at the programmer's discretion and ends up being explicit. But, your proposed syntax is not usable because it is ambiguous. Exactly what you propose is already used for decorators in a way that the parser would not be able to distinguish a decorator from an import expression. Consider: @ham.spam() # import expression or decorator? def eggs(): ... This currently parses as a decorator and for backwards compatibility that must not change, which means that import expressions would be usable in some contexts and not others purely based on what follows them. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IS7WHO3ZL33D4PSHWFSLU4D4WQ5736MY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)
On Sun, 14 Nov 2021, 19:07 Christopher Barker, wrote:
> On Sun, Nov 14, 2021 at 10:27 AM MRAB wrote:
>
>> Unfortunately, it goes too far, because it's unlikely that we want "ᵖ"
>> ("\N{MODIFIER LETTER SMALL P}') to be equivalent to "P" ("\N{LATIN
>> CAPITAL LETTER P}".
>>
>
> Is it possible to only capture things like the combining characters and
> not the "equivalent" ones like the above?
>
Yes, that is NFC. NKFC converts to equivalent characters and also composes;
NFC just composes.
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/YX5YJBQH4CIAF6GIIE7L54GGWLPAGVGB/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Having Sorted Containers in stdlib?
Serhiy Storchaka wrote: > From my C++ and Java experience, hashtable-based containers are much > more useful than tree-based containers. They are more compact and fast. > In most cases the only reason of using sorted container is supporting > deterministic iteration order, but often it is enough to sort data only > for output. The concern is not constant speed factor but asymptotic complexity. Inserting or deleting in a tree-based sorted collection is O(log n), but lists are O(n): the bisect is O(log n) but moving trailing list elements is O(n). It's memmove fast but dominates as n gets large. There are many algorithms that are hard to implement as fast as they can be without a tree. Something like rolling median would be a good test case, e.g. "for a list N containing numbers, produce a list M of length len(N) - 1 where M[i] == statistics.median(N[:i+1])". I think the best you can do with the standard library and without writing a balanced binary tree from scratch is O(n²) (if you can beat this I'd like to know how!). With a tree-based sorted list it's O(n log n). Dicts and sets cannot maintain an arbitrary order, except that OrderedDict.move_to_end() very occasionally turns out to be all you need (and even then, I find it tricky to reason about and apply correctly in an algorithm: it requires you to maintain an invariant, where SortedList maintains the sorted invariant for you). > There is no such need > of tree-based implementation. It is not needed in Python core, and is > not needed for most users. So it is good to keep it in third-party > libraries. Maintaining it in the stdlib has a cost and the benefit/cost > value would be low. The situation where this bites my firm the most is in interviews. This is exactly the situation where we're asking candidates to demonstrate their CS knowledge and produce an algorithm that has low asymptotic complexity. We use various services including HackerRank, which provide a sandboxed Python environment that cannot use PyPI. There are quite a few services in this kind of Educational+Challenge space including some that execute Python in the browser using Brython etc. My team believes that candidates with a Java background find the challenge easier than candidates with a Python background, because they just use NavigableSet and move on. I find that kind of embarrassing. It requires us to apologise for Python's deficiency and fudge how we score the question, which makes it hard to know we're providing a level playing field. (I would argue that we should not ask those kinds of questions or not use these sandboxed interview environments but institutional friction makes that magnitude of change very difficult.) Putting something in the standard library means that it will, in due course, appear in HackerRank and other services we use, and we can expect candidates to be familiar with it and incorporate it in their solutions. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VTOYBWO5OWXID4OIH5FWCKWSWRDXKSTB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Python multithreading without the GIL
On Fri, 8 Oct 2021 at 03:50, Sam Gross wrote: > My goal with the proof-of-concept is to demonstrate that removing the GIL is > feasible and worthwhile, and that the technical ideas of the project could > serve as a basis of such an effort. I'm a novice C programmer, but I'm unsure about the safety of your thread-safe collections description. You describe an algorithm for lock-free read access to list items as 1. Load the version counter from the collection 2. Load the “backing array” from the collection 3. Load the address of the item (from the “backing array”) 4. Increment the reference count of the item, if it is non-zero (otherwise retry) 5. Verify that the item still exists at the same location in the collection (otherwise retry) 6. Verify that the version counter did not change (otherwise retry) 7. Return the address of the item But you do the bounds check for the index before this, here[1]. If the thread is suspended after this and before you read the address of the backing array [2], the list could have been resized (shrunk), and the backing array reallocated from a new memory block. So the pointer you read at 3 could be from uninitialized memory that is beyond the size of the array (or within the array but larger than the current number of items). And then you write to it at 4 which is then a write into a random memory location. [1] https://github.com/colesbury/nogil/blob/fb6aabede5f7f1936a21c2f48ec7fcc0848d74bf/Objects/listobject.c#L137 [2] https://github.com/colesbury/nogil/blob/fb6aabede5f7f1936a21c2f48ec7fcc0848d74bf/Objects/listobject.c#L141 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6J6XFEACF2C6XPLZRVABUFFHJICUTZCS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: aiter/anext review request
As someone who was involved in implementing these, I think they should not be in builtins if that means they have to be in C. My argument is from a point of maintainability. Writing them was plenty of effort in the first place; Josh had written them in idiomatic async Python in the first place, my contribution was to unroll that to sync Python code, and then port that to (sync) C code. It was a lot of effort and a lot of code - several hundred lines and 4(?) new types. The Python code was a few lines - very readable and likely to be practically as fast. We weren't writing this in C to speed it up or to make the code better, but because we *had to*. Implementing async functionality in C is a pain because to implement an awaitable type you need not just that awaitable type, but a new type to represent the iterator that am_await returns. I could imagine having generic type objects and other helpers for implementing async PyObjects in C but I don't really envisage anyone doing that; if you want to write async helpers for Python the best framework is Python. As Josh can attest I was in two minds while implementing this change; I argued firstly that having them in the operator module is fine, and later, that if we want async builtins in general, maybe we could implement them in Python and freeze them into the binary. We pushed on with the C approach mostly because we were already 70% done, and this was what Yury asked for, so it seemed more likely that this would get merged. But, if we're still discussing whether this should be merged in builtins or operator, and that dictates whether it is in Python or C, I'm 100% behind having this code be Python. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VZFDUDH3NIBZX4ADPJ4E7VG2WAWOUBAA/ Code of Conduct: http://python.org/psf/codeofconduct/
