Re: [Python-ideas] if-statement in for-loop
On 5 October 2016 at 05:09, Ken Kundertwrote: > On Wed, Oct 05, 2016 at 03:07:42AM +1100, Steven D'Aprano wrote: >> >> Extra newlines are cheap. Writing >> > > The cost is paid in newlines *and* extra levels of indentation. No extra indentation if you ise "if not condition: continue" or refactor the condition into a custom iterable. Both of which have already been mentioned here as ways of achieving the desired result without a language change. > Why isn't it the programmer that is writing the code the best person to decide > what is best? Because the programmer writing the code isn't going to write and maintain the changes to the CPython/Jython/PyPy codebases, write the tests and documentation, support the questions that come from other users, etc...? More seriously, that argument could apply to *any* proposal. "Let the user decide whether to use the feature or not, and just add it". However, not all features get added precisely because someone has to make a cost/benefit judgement on any proposal and the people who do that are the CPython core devs. Discussion on this list is about thrashing out convincing arguments that will persuade the core devs - which is one of the reasons a lot of the core devs hang out here, to provide a sounding board on whether arguments are convincing or not. "Make the feature available and let the user decide if they want to use it" isn't a convincing argument. At best it could be a small part of a larger argument. It's countered by "does it make the language harder to teach having multiple ways of doing things?", "what about edge cases?" (in this case, trailing elses have been mentioned), "is there a well-known and easy workaround?", "no other languages (apart from Perl) seem to have this feature", ... and those issues need to be addressed in a full proposal. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] if-statement in for-loop
On Tue, Oct 04, 2016 at 09:09:40PM -0700, Ken Kundert wrote: > On Wed, Oct 05, 2016 at 03:07:42AM +1100, Steven D'Aprano wrote: > > > > Extra newlines are cheap. Writing > > > > The cost is paid in newlines *and* extra levels of indentation. You've quoted me out of context -- I did also refer to extra indentation being cheap. At the point that it isn't any more, it is a code smell and you (that's generic you, not just you personally) should think hard about how the design of your code. > Why isn't it the programmer that is writing the code the best person to > decide > what is best? Have you *seen* the quality of code written by the average coder? And remember, fifty percent of coders are worse than that. I jest, but only a bit. For better or worse, of course every programmer can set their own style, within the constraints of the language. If they cannot bear the language contraints, they're free to use a different language, or design their own. Anyone can be "the best person to decide" for their own private language. All languages have their own style, of what is or isn't allowed, what's encouraged and what's discouraged, and their own idiomatic way of doing things. The syntax constraints of the language depend on the language designer, not the programmers who use it. For some languages, those constraints are set by those who are appointed to sit on a standards board, usually driven by the corporations with the deepest pockets. Python, it is Guido and the core developers who set the boundaries of what coding styles can work in Python, and while the community can influence that, it doesn't control it. It isn't a wild free-for-all where every programmer is "the best person to decide". Some people might think that moving closer towards a Perl-ish one-liner culture by allowing (say): for x in seq for y in items if cond: block makes Python better ("saves some lines! saves some indents!"), but to those who like the discipline and structure of Python's existing loop syntax, this will make Python significantly worse. No decision can please everybody. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
Paul Moore wrote: I don't know *that* much about Erlang, but Python's model is that of a single shared address space with (potentially multiple) threads of code running, having access to that address space. I don't know much about Erlang either, but from what I gather, it's a functional language. That removes a lot of potential problems with concurrency right from the beginning. You can't have trouble with mutation of shared state if you can't mutate state in the first place. :-) -- Greg ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improve error message when missing 'self' in method definition
Hi all, A bit of shameless self-promotion but in case anyone interested, a while ago, I had started to work on a project to improve error message. In case anyone's interested, you can found everything at: https://github.com/SylvainDe/DidYouMean-Python . It can be invoked in different ways, one of them being a hook. For instance, you'd get something like: >>> import didyoumean_api >>> didyoumean_api.didyoumean_enablehook() >>> math.pi Traceback (most recent call last): File "", line 1, in NameError: name 'math' is not defined*. Did you mean to import math first?* There is still a lot to be done (and the main thing would be to make it pip installable) but it may be useful if the improved error messages do not make it to the CPython interpreter. Regards Sylvain ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
On 5 October 2016 at 16:49, Rene Nejsumwrote: >> On 04 Oct 2016, at 18:40, Sven R. Kunze wrote: >> I don't think that's actually what I wanted here. One simple keyword should >> have sufficed just like golang did. So, the developer gets a way to decide >> whether or not he needs it blocking or nonblocking **when using a >> function**. He doesn't need to decide it **when writing the function**. > > I agree, that’s why i proposed to put the async keyword in when creating the > object, saying in this instance I want asynchronous communication with the > object. OK, I think there may be a piece of foundational knowledge regarding runtime design that's contributing to the confusion here. Python's core runtime model is the C runtime model: threads (with a local stack and access to a global process heap) and processes (which contain a heap and one or more threads). Anything else we do (whether it's generators, coroutines, or some other form of paused execution like callback management) gets layered on top of that runtime model. When folks ask questions like "Why can't Python be more like Go?", "Why can't Python be more like Erlang?", or "Why can't Python be more like Rust?" and get a negative response, it's usually because there's an inherent conflict between the C runtime model and whatever piece of the Go/Erlang/Rust runtime model we want to steal. So the "async" keyword in "async def", "async for" and "async with" is essentially a marker saying "This is not a C-like runtime concept anymore!" (The closest C-ish equivalent I'm aware of would be Apple's Grand Central Dispatch in Objective-C and that shows many of the async/await characteristics also seen in Python and C#: https://www.raywenderlich.com/60749/grand-central-dispatch-in-depth-part-1 ) Go (as with Erlang before it) avoided these problems by not providing C-equivalent functions in the first place. Accordingly, *every* normal function defined in Go can also be used as a goroutine, rather than needing to be a distinct type - their special case is defining functions that interoperate with external C libraries. Python (along with other languages built on the C runtime model like C# and Objective-C) doesn't have that luxury - we need to distinguish coroutines from regular functions, since we can't just handle them according to the underlying C runtime model any more. Guido's idea of a shadow thread to let synchronous threads run coroutines without needing to actually run a foreground event loop should provide a manageable way of getting the two runtime models (traditional C and asynchronous coroutines) to play nicely together in a single application, and has the virtue of being something folks can readily experiment with for themselves before we commit to anything specific in the standard library (since all the building blocks of thread local storage, event loop management, and inter-thread message passing primitives are already available). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improve error message when missing 'self' in method definition
+1 I've definitely seen a lot of new users make this error, an improved message could go a long way. On Tue, Oct 4, 2016 at 1:52 PM, Nathan Goldbaumwrote: > Hi all, > > Recently pypy received a patch that improves the error message one gets > when 'self' is missing in a method's signature: > > https://mail.python.org/pipermail/pypy-dev/2016-September/014678.html > > Here are the commits that implement the change in pypy: > > https://bitbucket.org/pypy/pypy/commits/all?search= > branch(better-error-missing-self) > > I'm curious whether a similar improvement would also be received well in > CPython. In particular, this guides one to the correct solution for a > common programming mistake made by newcomers (and even not-so-newcomers). > > Here is a case study that found this was a common source of errors for > newcomers: > > http://dl.acm.org/citation.cfm?id=2960327 > > -Nathan > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improve error message when missing 'self' in method definition
+∞ Another long-time user here who occasionally still makes this mistake. Stephan 2016-10-05 19:29 GMT+02:00 Paul Moore: > On 5 October 2016 at 18:17, Lisa Roach wrote: > > +1 > > > > I've definitely seen a lot of new users make this error, an improved > message > > could go a long way. > > I'm not a new user by any means, and I still regularly make this > mistake. Because I've got the experience, I recognise the error when I > see it, but that's not much help for people who haven't already made > the mistake hundreds of times :-) > > +1 on improving the message. > Paul > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] if-statement in for-loop
On 5 October 2016 at 17:26, Nick Coghlanwrote: > Compared to those, locally modifying the token stream to inject ": > INDENT" pairs when the if and for keywords are encountered between an > opening "for" keyword and a closing ":" keyword would be a relatively > straightforward change that only impacted folks that decided they > preferred that particular flavour of Abbreviated Python to the regular > version. It's also worth noting that the obvious response "but I don't want to have to run a preprocessor against my code" is another indication that this isn't solving a significant enough problem to warrant a language change. Again, this isn't a hard and fast rule, but it is a useful rule of thumb - how much effort are you willing to go to to get this feature without it being built in? That's one of the reasons "it should be made into a module on PyPI" is a useful counter to proposals for new stdlib functions. It's also worth looking at the cases where things get added despite not going via that route - sometimes "being built in" is an important benefit of itself. But typically that's because people are encouraged to use built in facilities, so guiding beginners (or not-so-beginners) into good practice is important. In this case, it's far from clear that the feature is actually good practice. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improve error message when missing 'self' in method definition
On 5 October 2016 at 18:17, Lisa Roachwrote: > +1 > > I've definitely seen a lot of new users make this error, an improved message > could go a long way. I'm not a new user by any means, and I still regularly make this mistake. Because I've got the experience, I recognise the error when I see it, but that's not much help for people who haven't already made the mistake hundreds of times :-) +1 on improving the message. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improve error message when missing 'self' in method definition
On Thu, Oct 6, 2016 at 5:27 AM, Michel Desmoulinwrote: > +1. Python does need better error messages. This and the recent new import > exception will really help. > > Will feature freeze prevent this to get into 3.6 if some champion it? > Given that it's not changing semantics at all, just adding info/hints to an error message, it could well be added in a point release. +1 on any feature that helps people to debug code. This doesn't look overly spammy or anything, and it's easy for someone coming from C++ to forget to include that key parameter. ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
On the other hand, await / async is a fantastic interface to unify all concurrent paradigms and asyncio already provide a bridge with threads and subprocess. So it kinda make sense. Le 04/10/2016 à 18:40, Sven R. Kunze a écrit : On 04.10.2016 13:30, Nick Coghlan wrote: What it *doesn't* do, and what you need greenlet for, is making that common interface look like you're using plain old synchronous C threads. If folks really want to do that, that's fine - they just need to add gevent/greenlet as a dependency, just as the folks that don't like the visibly object-oriented nature of the default unittest and logging APIs will often opt for third party alternative APIs that share some of the same underlying infrastructure. Maybe, this is all a big misunderstanding. asyncio is incompatible with regular execution flow and it's **always blocking**. However, asyncio is perceived by some of us (including me) as a shiny alternative to processes and threads but really isn't. I remember doing this survey on python-ideas (results here: https://srkunze.blogspot.de/2016/02/concurrency-in-python.html) but I get the feeling that we still miss something. My impression is that asyncio shall be used for something completely different than dropping off things into a background worker. But looking at the cooking example given by Steve Dower (cf. blog post), at other explanations, at examples in the PEPs, it just seems to me that his analogy could have been made with threads and processes as well. At its core (the ASYNC part), asyncio is quite similar to threads and processes. But its IO-part seem to drive some (design) decisions that don't go well with the existing mental model of many developers. Even PEP-reviewers are fooled by simple asyncio examples. Why? Because they forget to spawn an eventloop. "async def and await" are just useless without an eventloop. And maybe that's what's people frustration is about. They want the ASYNC part without worrying about the IO part. Furthermore, adding 2 (TWO) new keywords to a language has such an immense impact. Especially when those people are told "the barrier for new keywords is quite high!!". So, these new keywords must mean something. I think what would help here are concrete answers to: 0) Is asyncio a niche feature only be used for better IO? 1) What is the right way of integrating asyncio into existing code? 2) How do we intend to solve the DRY-principle issue? If the answer is "don't use asyncio", that's a fine result but honestly I think it would be just insane to assume that we got all these features, all this work and all those duplicated functions all for nothing. I can't believe that. So, I am still looking for a reasonable use-case of asyncio in our environment. Cheers, Sven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improve error message when missing 'self' in method definition
On 5 October 2016 at 20:55, Yury Selivanovwrote: > > Speaking of, I'm not much of a C hacker, and messing with CPython internals >> is a little daunting. If anyone wants to take this on, you have my >> blessing. I also may take a shot at implementing this idea in the next >> couple weeks when I have some time. >> > > It would help if you could create an issue and write exhaustive unittests > (or at least specifying how exactly the patch should work for all corner > cases). Someone with the knowledge of CPython internals will later add the > missing C code to the patch. > > Yury > > I agree with Yury here. There are corner cases (like what to do with classmethods etc). If behaviour for all of them are specified, it would be quite straightforward to implement this. -- Ivan ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
On 05.10.2016 20:23, Michel Desmoulin wrote: On the other hand, await / async is a fantastic interface to unify all concurrent paradigms and asyncio already provide a bridge with threads and subprocess. So it kinda make sense. Almost if it would not require duplicate pieces of code. But maybe, we are wrong and there won't be any duplications. Cheers, Sven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] xfork [was Re: async objects]
On 05.10.2016 06:11, Ethan Furman wrote: On 10/04/2016 09:40 AM, Sven R. Kunze wrote: As a result of past discussions, I wrote the module "xfork" which basically does this "golang goroutine" stuff. It's just a thin wrapper around "futures" but it allows to avoid that what René and Anthony objects about. Looks cool! Thanks! You're welcome. :) Cheers, Sven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Improve error message when missing 'self' in method definition
Isn't it possible to implement it as a pure Python exception hook? On Wed, Oct 5, 2016 at 10:04 PM Ivan Levkivskyiwrote: > > On 5 October 2016 at 20:55, Yury Selivanov > wrote: > > > Speaking of, I'm not much of a C hacker, and messing with CPython internals > is a little daunting. If anyone wants to take this on, you have my > blessing. I also may take a shot at implementing this idea in the next > couple weeks when I have some time. > > > It would help if you could create an issue and write exhaustive unittests > (or at least specifying how exactly the patch should work for all corner > cases). Someone with the knowledge of CPython internals will later add the > missing C code to the patch. > > Yury > > > I agree with Yury here. There are corner cases (like what to do with > classmethods etc). If behaviour for all of them are specified, it would be > quite straightforward to implement this. > > -- > Ivan > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
On 05.10.2016 08:49, Rene Nejsum wrote: As a result of past discussions, I wrote the module "xfork" which basically does this "golang goroutine" stuff. It's just a thin wrapper around "futures" but it allows to avoid that what René and Anthony objects about. I had a look at xfork, and really like it. It is implemented much like the lower level of PYWORKS and PYWORKS could build on xfork instead. Thanks. :) I think that the “model” of doing async should be defined in the Python language/runtime (like in Go, Erlang, ABCL) . I the ideal case it should be up to the runtime implementation (CPython, PyPy, Jython, IronPython etc.) how the asynchronous behaviour is implemented (greenlets, threads, roll-it-own, etc) That's the way I see it as well. The Python language is extremely high-level. So, I guess in most cases, most people would just use the default implementation. Cheers, Sven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
On 10/05/2016 12:20 PM, Sven R. Kunze wrote: On 05.10.2016 18:06, Nick Coghlan wrote: Guido's idea of a shadow thread to let synchronous threads run coroutines without needing to actually run a foreground event loop should provide a manageable way of getting the two runtime models (traditional C and asynchronous coroutines) to play nicely together in a single application, and has the virtue of being something folks can readily experiment with for themselves before we commit to anything specific in the standard library (since all the building blocks of thread local storage, event loop management, and inter-thread message passing primitives are already available). I needed to think about this further when Guido mentioned it. But I like it now. If you check https://github.com/srkunze/fork/tree/asyncio , I already started working on integrating asyncio into xfork at long time ago. But I still couldn't wrap my mind around it and it stalled. But IIRC, I would have implemented a shadow thread solution as well. So, if his idea goes into the stdlib first, I welcome it even more as it would do the heavy lifting for me. xfork would then be just a common interface to threads, processes and coroutines. At this point I'm willing to bet that you (Sven) are closest to actually having a shadow thread thingy that actually works. Maybe some other asyncio folks would be willing to help you develop it? -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
> On 05 Oct 2016, at 21:20, Sven R. Kunzewrote: > > On 05.10.2016 18:06, Nick Coghlan wrote: >> [runtime matters] > > I think I understand your point. > > I also hope that others and me could provide you with our perspective. We see > Python not as a C-like runtime but as an abstract modelling language. I know > that it's different from the point of view of CPython internals, however from > the outside Python suggests to be much more than a simple wrapper around C. > Just two different perspectives. Excellent point. For me CPython, Jython, IronPython, PyPy are the same (99.9%) and the important part is the Python the language. For a long time I tested PYWORKS again all implementations and were happy that it ran on all. Clearly, for others CPython (incl. runtime and C-bindings) is the fact and the others are far from the same, especially because the missing C-integration. But, are the runtimes for Python and Erlang that fundamentally different? Is it Python’s tight integration with C that is the big difference? When I first read about the async idea, I initially expected that it would be some stackless like additions to Python. My wish for Python was an addition to the language the allowed an easy an elegant concurrent model on the language level. Ideally a Python program with 1000 async objects parsing a 10TB XML in-memory file, should run twice as fast on a 8-core CPU, compared to a 4-core ditto. > Unfortunately, your runtime explanations still don't address the DRY issue. > :-/ > >> Guido's idea of a shadow thread to let synchronous threads run >> coroutines without needing to actually run a foreground event loop >> should provide a manageable way of getting the two runtime models >> (traditional C and asynchronous coroutines) to play nicely together in >> a single application, and has the virtue of being something folks can >> readily experiment with for themselves before we commit to anything >> specific in the standard library (since all the building blocks of >> thread local storage, event loop management, and inter-thread message >> passing primitives are already available). > > I needed to think about this further when Guido mentioned it. But I like it > now. > > If you check https://github.com/srkunze/fork/tree/asyncio , I already started > working on integrating asyncio into xfork at long time ago. But I still > couldn't wrap my mind around it and it stalled. But IIRC, I would have > implemented a shadow thread solution as well. So, if his idea goes into the > stdlib first, I welcome it even more as it would do the heavy lifting for me. > xfork would then be just a common interface to threads, processes and > coroutines. xfork (as pyworks) implements a proxy object, which “almost” behaves like the real object, but it is still a proxy. If fork (or spawn, chan, async, whatever.) was a part of the language it would be more clean. br /Rene > > Cheers, > Sven > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] async objects
On 5 October 2016 at 21:28, Rene Nejsumwrote: > But, are the runtimes for Python and Erlang that fundamentally different? Is > it Python’s tight integration with C that is the big difference? I don't know *that* much about Erlang, but Python's model is that of a single shared address space with (potentially multiple) threads of code running, having access to that address space. Erlang's model is that of multiple threads of execution (processes) that are isolated from each other (they have independent address spaces). That's a pretty fundamental difference, and gets right to the heart of why async is fundamentally different in the two languages. It also shows in Erlang's C FFI, which as I understand it is to have the C code isolated in a separate "process", and the user's program communicating with it through channels. As far as I can see, that's a direct consequence of the fact that you couldn't safely expect to call a C function (with its direct access to the whole address space) direct from an Erlang process. Python's model is very similar to C (and Java, and C#/.net, and many other "traditional" languages [1]). That's not "to make it easier to call C functions", it's just because it was a familiar and obvious model to use, known to work well, when Python was first developed. The fact that it made calling C from Python easy was a side effect - one that helped make Python as useful and popular as it is today, but nevertheless simply a side effect of the model. Paul [1] And actual computer hardware, which isn't a coincidence :-) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/