Re: how to discover what values produced an exception?
On Tue, 7 May 2024 at 03:38, Alan Bawden via Python-list wrote: > A good error message shouldn't withhold any information that can > _easily_ be included. Debugging is more art than science, so there is > no real way to predict what information might prove useful in solving > the crime. I emphasized "easily" because of course you have to draw the > line somewhere. Emphasis on "easily" is correct here. How easy is it to report the value of something that could be arbitrarily large? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: how to discover what values produced an exception?
From a practical perspective: not all values are printable (especially if printing a value results in an error: then you'd lose the original error, so, going crazy with printing of errors is usually not such a hot idea). But, if you want the values: you'd have to examine the stack, extract the values from the local variables etc. It's easier to do this interactively (unless you are in a multithreaded environment, where pdb is broken). But, you could also try to find this information automatically (by unwinding the stack to the place that generated the error, examining the local variables etc.) It's tedious and prone to errors. So, if you really want to do this automatically for every error that's going to be quite a bit of work. On Fri, May 3, 2024 at 6:58 PM Johanne Fairchild via Python-list wrote: > > How to discover what values produced an exception? Or perhaps---why > doesn't the Python traceback show the values involved in the TypeError? > For instance: > > --8<>8--- > >>> (0,0) < 4 > Traceback (most recent call last): > File "", line 1, in > TypeError: '<' not supported between instances of 'tuple' and 'int' > --8<>8--- > > It could have said something like: > > --8<>8--- > TypeError: '<' not supported between instances of 'tuple' and 'int' > in (0,0) < 4. > --8<>8--- > > We would know which were the values that caused the problem, which would > be very helpful. > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: how to discover what values produced an exception?
Thomas Passin writes: On 5/3/2024 9:56 AM, Johanne Fairchild via Python-list wrote: > How to discover what values produced an exception? Or perhaps---why > doesn't the Python traceback show the values involved in the TypeError? > For instance: > > --8<>8--- >>>> (0,0) < 4 > Traceback (most recent call last): >File "", line 1, in > TypeError: '<' not supported between instances of 'tuple' and 'int' > --8<>8--- > > It could have said something like: > > --8<>8--- > TypeError: '<' not supported between instances of 'tuple' and 'int' >in (0,0) < 4. > --8<>8--- > > We would know which were the values that caused the problem, which would > be very helpful. In this example it would not help at all to know the actual values. Knowing that you are trying to compare incomparable types is enough. In general, it absolutely can help. The programmer can sometimes recognize where a value of unexpected type came from just by looking at it, allowing her to quickly deduce just what went wrong without further investigation. A good error message shouldn't withhold any information that can _easily_ be included. Debugging is more art than science, so there is no real way to predict what information might prove useful in solving the crime. I emphasized "easily" because of course you have to draw the line somewhere. The fact that Python error messages often fail to mention the actual objects that caused the error has always annoyed me. I've always presumed that for some reason it just wasn't easy to do. And it's never been more than a minor annoyance to me. So the OP is not wrong for wishing for this. Other programming languages do it. Other Python programmers miss it. - Alan -- https://mail.python.org/mailman/listinfo/python-list
Re: how to discover what values produced an exception?
On 2024-05-03 at 10:56:39 -0300, Johanne Fairchild via Python-list wrote: > How to discover what values produced an exception? Or perhaps---why > doesn't the Python traceback show the values involved in the TypeError? > For instance: > > --8<>8--- > >>> (0,0) < 4 > Traceback (most recent call last): > File "", line 1, in > TypeError: '<' not supported between instances of 'tuple' and 'int' > --8<>8--- > > It could have said something like: > > --8<>8--- > TypeError: '<' not supported between instances of 'tuple' and 'int' > in (0,0) < 4. > --8<>8--- > > We would know which were the values that caused the problem, which would > be very helpful. I'm not disagreeing that knowing the values could be useful in many cases. In the general case, though, it's not practical. Consider a function like this: def f(x, y): return g(x) < h(y) The printed values of x, y, g(x), and h(y) could all be millions of (or more) glyphs. Worse, one or more of those values could contain circular lists or similar structures. And h or g could have changed x or y. In summary, printing run-time values isn't always safe or useful. At least printing the types is safe. In the face of ambiguity, refuse to guess. -- https://mail.python.org/mailman/listinfo/python-list
Re: how to discover what values produced an exception?
Johanne Fairchild wrote at 2024-5-3 10:56 -0300: >How to discover what values produced an exception? Or perhaps---why >doesn't the Python traceback show the values involved in the TypeError? >For instance: > >--8<>8--- >>>> (0,0) < 4 >Traceback (most recent call last): > File "", line 1, in >TypeError: '<' not supported between instances of 'tuple' and 'int' >--8<>8--- > >It could have said something like: > >--8<>8--- >TypeError: '<' not supported between instances of 'tuple' and 'int' > in (0,0) < 4. >--8<>8--- > >We would know which were the values that caused the problem, which would >be very helpful. Typically, the traceback informs you about the source code line where the exception has been raised. When the source line contains literals, you see the values. If not, then only in special (speak: rather simple) cases the knowledge of the values will help much. Usually, a more thorough analysis is required to find out the bug location and how to fix the bug. -- https://mail.python.org/mailman/listinfo/python-list
Re: how to discover what values produced an exception?
On 5/3/2024 9:56 AM, Johanne Fairchild via Python-list wrote: How to discover what values produced an exception? Or perhaps---why doesn't the Python traceback show the values involved in the TypeError? For instance: --8<>8--- (0,0) < 4 Traceback (most recent call last): File "", line 1, in TypeError: '<' not supported between instances of 'tuple' and 'int' --8<>8--- It could have said something like: --8<>8--- TypeError: '<' not supported between instances of 'tuple' and 'int' in (0,0) < 4. --8<>8--- We would know which were the values that caused the problem, which would be very helpful. In this example it would not help at all to know the actual values. Knowing that you are trying to compare incomparable types is enough. -- https://mail.python.org/mailman/listinfo/python-list
how to discover what values produced an exception?
How to discover what values produced an exception? Or perhaps---why doesn't the Python traceback show the values involved in the TypeError? For instance: --8<>8--- >>> (0,0) < 4 Traceback (most recent call last): File "", line 1, in TypeError: '<' not supported between instances of 'tuple' and 'int' --8<>8--- It could have said something like: --8<>8--- TypeError: '<' not supported between instances of 'tuple' and 'int' in (0,0) < 4. --8<>8--- We would know which were the values that caused the problem, which would be very helpful. -- https://mail.python.org/mailman/listinfo/python-list
CoC Warning [was: What is Install-Paths-To in WHEEL file?]
On 12/29/23 05:02, Left Right via Python-list wrote: Wow. That place turned out to be the toxic pit I didn't expect. It's a shame that a public discussion of public goods was entrusted to a bunch of gatekeepers with no sense of responsibility for the thing they keep the keys to. Personal attacks are not welcome nor tolerated. Please be constructive when engaging with the community. -- ~Ethan~ -- https://mail.python.org/mailman/listinfo/python-list
[dead thread] Re: What is Install-Paths-To in WHEEL file?
This thread is no longer being useful, and is now closed. -- ~Ethan~ Moderator -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
> others do not and so your notion of what is "accepted" > is not universally shared. Why should I or anyone else care about what "others" think? The important question is whether what I do is right. And the answer is "yes". That's why there are rules in the first place instead of polling. > if you want to influence anything Usually, when I interact with representatives of Python community I have two goals: 1. Typically, I need to show to someone who's paying my salary why something produced by this community doesn't work. I.e. say, I need to convince a project manager on a project I'm helping maintain that deploying using "pip install" is a bad idea. I write an explanation which I share with the PM and the PyPA people in the bug tracker. They predictably block me out of fear or frustration. This gives me a proof that the thing doesn't work (well), and I'm allowed to do it the right way. Just like in your previous remark: majority could be a good initial heuristic, but proof is still a lot better. 2. At this point, I have no hope of convincing the prominent members of Python community how awful a lot of their decisions are. There are plenty of socially constructed obstacles on this way. The reason I do this is posterity. There are plenty of people who aren't influenced by the internal developments of Python community (outside of it) and they can see much of its development for what it is: commenting on this development honestly will help them make an informed choice. It's also important that those who will come after us will learn about this contradiction. Too many bad projects with bad design outlived their good counterparts due to popularity caused by chance. And today those better design and ideas are as good as lost. For example, Unix outlived and "overpowered" plenty of better operating systems of its time. But most programmers today would have no idea what those systems were and how they were different. Similarly, x86 ISA. And plenty more. Python changed from its early days of trying to be funny and generally welcoming of many contradicting ideas and opinions into a Lord of the Flies community that no longer tolerates differences of opinion. It's lost the spirit of "playful cleverness" (as RMS would put it), and became a "don't think, do as I say" community. I want to make sure those who come to learn about Python will not miss this aspect of its history. -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sun, 31 Dec 2023 at 00:35, Left Right wrote: > > It's not for you to choose the way I communicate. There are accepted > boundaries, and I'm well within those boundaries. Anything beyond that > is not something I'm even interested in hearing your opinion on. You might not be interested in my opinion but you might want to reflect on the fact that although you consider your behavior to be within "accepted boundaries" the evidence here (and in the forum) suggests that others do not and so your notion of what is "accepted" is not universally shared. I am not going to reply to your other points except to say that if you want to influence anything then I expect that you would have more success with a different approach. To anyone else considering replying in this thread: please don't. I very much doubt that anything good will happen here. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
> You are conflating several different groups of people. The PyPA are > the people who currently maintain the code for various > libraries/tools. That is very often not the same as the people who > originally wrote the code for the same libraries/tools or for > preceding ones. Neither group is the same as the forum moderators (I > suspect that there is no intersection between the moderators and the > PyPA etc.). I'm sorry to tell you, but you suspect wrong. Unfortunately, it's always the same story. Whenever I or anyone else with a legitimate complaint about larger projects managed by PyPA tries to bring this to public discussion, they get banned and their comments about PyPA activity removed. It's always presented as if whoever is complaining is disrespecting the hard work of the person replying, who usually self-describe as selfless volunteer with limited time and attention they are willing to grant to the one complaining (implying they either are PyPA or are volunteering for them). As if their time was obviously more important than that was spent by the one complaining to research the problem and put together the complaint. This has nothing to do with the original authors of the projects managed by PyPA. I don't know why you decided to bring this up. I haven't mentioned them. > Actually you are wasting the time of others by putting across > inaccurate and unhelpful information in a rude way and at the same > time criticising others without really understanding who you are > criticising and for what. Your contribution is unhelpful mostly (but > not exclusively) because of the way that you choose to communicate. No, I'm not _wasting_ anyone's time. I bring up a legitimate issue that needs solving. What happens is a typical example of gatekeeping, overestimating one's worth or the value of one's contribution. The time I had to waste because of the bad decisions made by PyPA is orders of magnitude more than the time they have spent reading whatever I wrote to them. Point me to inaccurate information please. I'm happy to be corrected. Similarly, point me to where I was rude, and I will apologize. Apparently, I have a better understanding of who I criticize and for what than you do. You need to at least be factual when you make these sorts of claims. It's not for you to choose the way I communicate. There are accepted boundaries, and I'm well within those boundaries. Anything beyond that is not something I'm even interested in hearing your opinion on. > There is some significant irony in you describing the forum as a > "toxic pit" for deleting your posts. I don't always agree with the > moderators and I am not sure that I would have reacted the way that > they did but these threads remind me precisely why moderation > (including deleting posts such as yours) is needed to *prevent* a > forum from turning into a toxic pit. You, as well as the moderators of the toxic pit forum are confused about what it means to have a good discussion. The discussion that is currently happening around PyPA projects and ideas is broken because the PyPA side of the discussion is unwilling to acknowledge how bad they are at doing their job. Whenever any serious criticism of their work surfaces, they deal with it by deleting the criticism, never through addressing the problem. You can be the most polite and humble person in the world, but as soon as you bring up the subject of the quality of their decisions, you are automatically excluded from discussion. The only criticism anyone is allowed to have is the kind that doesn't touch on any major projects. It's possible to point out typos in documentation or to address similarly inconsequential defects at the smaller code unit level, but it's not possible to call for a revision of ideas behind libraries or PEPs. For instance, as soon as you mention the comically awful idea of pyproject.toml in a bad light, you get a ban. I believe this comes from the place of insecurity in one's ideas, and has nothing to do with how polite the criticism is. And that's when instruments like "code of conduct" are called upon to delete the inconvenient criticism. This is what creates toxic communities like StackOverflow or similarly built social networks which endow their moderators with way too much power over other users. The other extreme of anarchy, similar to 4chan, doesn't suffer from this problem, but sometimes results in grotesque gore or other _unpleasant_ things but aren't toxic in the same way gatekeeping is. This is how I understand and use the word "toxic". The dicuss.python.org is just as toxic as StackOverflow -- I don't have a metric precise enough to tell who's worse. I believe that this format is a very unfortunate choice for public discussion where there isn't an inherent division between owners and non-owners. Where giving the keys to the common good to a small group of people creates such a division. -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Fri, 29 Dec 2023 at 22:38, Left Right via Python-list wrote: > > > Then your understanding is flat-out wrong. Encouraging participation > > by everyone DOES mean deleting what is unproductive, offensive, and > > likely to discourage participation. > > I haven't written anything unproductive or offensive. I offered > constructive criticism with a detailed plan on how to fix the problem. > The forum owners chose to ban me because they don't like hearing that > the code they've written is bad. And that's the long and the short of > it. This has been a pattern in behavior of PyPA members I've > interacted with so far. You are conflating several different groups of people. The PyPA are the people who currently maintain the code for various libraries/tools. That is very often not the same as the people who originally wrote the code for the same libraries/tools or for preceding ones. Neither group is the same as the forum moderators (I suspect that there is no intersection between the moderators and the PyPA etc.). > And whenever they had a chance, they'd use it > to pretend that the problems I'm talking about don't exist by deleting > every mention of the problem. That is an example of unproductive and > offensive behavior because it produces nothing and wastes my time I've > dedicated to locating, reporting and solving their problem. Actually you are wasting the time of others by putting across inaccurate and unhelpful information in a rude way and at the same time criticising others without really understanding who you are criticising and for what. Your contribution is unhelpful mostly (but not exclusively) because of the way that you choose to communicate. I did not realise earlier what you were referring to but I see now that I have email notifications with the content of your posts that were deleted. I am not surprised that they were deleted and that you were banned because if I was a moderator looking at those then I would not expect a promising future for your interactions with others in the forum. There is some significant irony in you describing the forum as a "toxic pit" for deleting your posts. I don't always agree with the moderators and I am not sure that I would have reacted the way that they did but these threads remind me precisely why moderation (including deleting posts such as yours) is needed to *prevent* a forum from turning into a toxic pit. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
> Then your understanding is flat-out wrong. Encouraging participation > by everyone DOES mean deleting what is unproductive, offensive, and > likely to discourage participation. I haven't written anything unproductive or offensive. I offered constructive criticism with a detailed plan on how to fix the problem. The forum owners chose to ban me because they don't like hearing that the code they've written is bad. And that's the long and the short of it. This has been a pattern in behavior of PyPA members I've interacted with so far. And whenever they had a chance, they'd use it to pretend that the problems I'm talking about don't exist by deleting every mention of the problem. That is an example of unproductive and offensive behavior because it produces nothing and wastes my time I've dedicated to locating, reporting and solving their problem. > Go play in your own sandbox somewhere, You are being repeatedly rude, without provocation, and yet you keep blaming me for what you are doing. I guess you have to be a moderator in this forum because you act as if this is a kind of behavior will be without any repercussions for you. You probably don't understand it, but this sandbox is as much yours as it is mine. You can "become" an authority and, eg. block me -- but that would be an overreach. Physically possible but morally wrong. I don't need to prove you wrong by being better than you. Nobody does. Being right or wrong isn't about being better at something. Not only that, I legally (and physically) cannot establish my own Python Software Foundation and claim a right to Python intellectual property, establish a governing body for Python etc. These forums are how PSF is supposed to implement its advertised policies. I cannot just take over them... that'd be illegal even if I somehow managed to physically pull it off. -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sat, 30 Dec 2023 at 06:58, Left Right wrote: > My understanding is that "welcome and encourage participation by > everyone" is in stark contradiction to banning someone disagreeing > with you. Then your understanding is flat-out wrong. Encouraging participation by everyone DOES mean deleting what is unproductive, offensive, and likely to discourage participation. Your entire argument is based on misconceptions. Go play in your own sandbox somewhere, see if you can make something where everyone is welcome, including the toxic AND the people who dislike toxicity. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
Previously you wrote: > Here is the discussion referred to: https://discuss.python.org/t/what-is-install-paths-to-in-wheel-file/42005 This illustrates you had no idea what the discussion was about and now you write: > Oh trust me, I saw the discussion previously. Both cannot be true at the same time, unless you had some kind of very brief memory loss. > I'm not a lawyer, Neither am I. All I have to work with is my understanding of the English language. Here's how I come to my conclusions. The Python license grants all intellectual rights to Python to PSF (an American NGO, a.k.a. 501(c) organization), which, essentially, can be characterized as an organization for public good. This is what it has to say about itself in its mission statement: > Mission > The mission of the Python Software Foundation is to promote, protect, > and advance the Python programming language, and to support and > facilitate the growth of a diverse and international community of Python > programmers. it also elaborates what it means by "diverse" as follows: > Diversity > The Python Software Foundation and the global Python community > welcome and encourage participation by everyone. Our community > is based on mutual respect, tolerance, and encouragement, and we > are working to help each other live up to these principles. We want > our community to be more diverse: whoever you are, and whatever > your background, we welcome you. My understanding is that "welcome and encourage participation by everyone" is in stark contradiction to banning someone disagreeing with you. Note, I haven't offended anyone. I haven't even spoken to anyone who found themselves being offended. All I did was to describe in some detail the reasons why some projects endorsed by PyPA are a bad idea. You, as well as anyone else, are welcome to believe differently. This is the whole point of diversity allegedly promoted by PSF. I will think you are wrong, but it's not my place to shut you up. Neither is it the place of people in charge of the public discussion of Python or its satellite projects. They are not there to decide who's right and who gets the stage. Their role is to preserve the public good, which any discussion about subjects relevant to Python would be. What happens, however, and this is the unfortunate fate of popular projects, is that a small group of people consolidate all means of control in their hands, and the more control they get, the easier it is to get even more of it. The natural factor that would prevent this from happening: the community dissatisfaction with their role becomes increasingly less powerful as soon as more and more members of the community come to depend on the good provided by the community. If this discuss.python.org is representative of the Python community as a whole, then, unfortunately, it means that the goals PSF set for it are fading into the distance, rather than becoming more attainable. -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sat, 30 Dec 2023 at 01:37, Left Right wrote: > > > Yeah, because you have the God-given RIGHT to be able to say anything > > you like, on anyone's web site, and nobody's allowed to delete > > anything you say! That's how it goes, right? > > I don't believe in god, and I don't believe he / she can give me > rights. What I believe in is that Python is a public good, and its > status is enshrined in the license it uses. Is it? I'm not a lawyer, but I really don't think that that's what the license entitles you to. Can you quote the relevant parts of it? > > Don't let the door hit you on the way out. > > Oh, great. Here we go again. You don't even know what this discussion > is about, but decided to be rude. Oh trust me, I saw the discussion previously. I know what it is about. And when it comes to rudeness, let's just say, you reap what you sow. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
> Yeah, because you have the God-given RIGHT to be able to say anything > you like, on anyone's web site, and nobody's allowed to delete > anything you say! That's how it goes, right? I don't believe in god, and I don't believe he / she can give me rights. What I believe in is that Python is a public good, and its status is enshrined in the license it uses. I also believe that Python Foundation and PyPA are the public bodies that are meant to, beside other things, make sure that the public good stays that way. Me, being a member of the public, for whom the good is mean, means I have a right to discuss, complain or argue about the nature or function of this good. I, or you, or anyone else don't need god to make this happen. The rights I'm talking about are a consequence of the license that governs Python and various satellite projects. > Don't let the door hit you on the way out. Oh, great. Here we go again. You don't even know what this discussion is about, but decided to be rude. I mean, you don't have to be curious, and there's no need for you to try to figure out what this is about, but being rude without provocation? Just why? What do you stand to gain from this? -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Sat, 30 Dec 2023 at 01:16, Left Right via Python-list wrote: > > That's not the discussion that was toxic. But the one that was -- > doesn't exist anymore since the forum owners deleted it. > > The part where the forum owners delete whatever they disagree with is > the toxic part. Yeah, because you have the God-given RIGHT to be able to say anything you like, on anyone's web site, and nobody's allowed to delete anything you say! That's how it goes, right? You're most welcome to avoid the Python Discourse if you dislike moderated forums, but do be aware that python-list is ALSO moderated, and that completely unmoderated forums are far far more toxic than anything I've seen on the Python Discourse. Don't let the door hit you on the way out. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
That's not the discussion that was toxic. But the one that was -- doesn't exist anymore since the forum owners deleted it. The part where the forum owners delete whatever they disagree with is the toxic part. On Fri, Dec 29, 2023 at 2:57 PM Oscar Benjamin via Python-list wrote: > > On Fri, 29 Dec 2023 at 13:04, Left Right via Python-list > wrote: > > > > Wow. That place turned out to be the toxic pit I didn't expect. > > > > It's a shame that a public discussion of public goods was entrusted to > > a bunch of gatekeepers with no sense of responsibility for the thing > > they keep the keys to. > > Here is the discussion referred to: > https://discuss.python.org/t/what-is-install-paths-to-in-wheel-file/42005 > > I don't see anything "toxic" in that discussion. You asked questions > and people took the time to give clear answers. > > The basic answer to your question is that PEP 491 was never completed > and so there is no accepted specification of the Install-Paths-To > feature that it had been intended to introduce. The PEP text itself is > reasonably clear about this and also links to the up to date > specifications: > https://peps.python.org/pep-0491/#pep-deferral > > Instead for understanding the wheel format the appropriate document is: > https://packaging.python.org/en/latest/specifications/binary-distribution-format/ > > That document does not mention Install-Paths-To because it documents > the standards as defined and accepted via the PEP process but PEP 491 > was never accepted. > > -- > Oscar > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
On Fri, 29 Dec 2023 at 13:04, Left Right via Python-list wrote: > > Wow. That place turned out to be the toxic pit I didn't expect. > > It's a shame that a public discussion of public goods was entrusted to > a bunch of gatekeepers with no sense of responsibility for the thing > they keep the keys to. Here is the discussion referred to: https://discuss.python.org/t/what-is-install-paths-to-in-wheel-file/42005 I don't see anything "toxic" in that discussion. You asked questions and people took the time to give clear answers. The basic answer to your question is that PEP 491 was never completed and so there is no accepted specification of the Install-Paths-To feature that it had been intended to introduce. The PEP text itself is reasonably clear about this and also links to the up to date specifications: https://peps.python.org/pep-0491/#pep-deferral Instead for understanding the wheel format the appropriate document is: https://packaging.python.org/en/latest/specifications/binary-distribution-format/ That document does not mention Install-Paths-To because it documents the standards as defined and accepted via the PEP process but PEP 491 was never accepted. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
Wow. That place turned out to be the toxic pit I didn't expect. It's a shame that a public discussion of public goods was entrusted to a bunch of gatekeepers with no sense of responsibility for the thing they keep the keys to. On Wed, Dec 27, 2023 at 9:49 PM Left Right wrote: > > Thanks. I tried asking there. > > On Sun, Dec 24, 2023 at 11:53 PM Barry wrote: > > > > > > > > On 24 Dec 2023, at 00:58, Left Right via Python-list > > wrote: > > > > I'm trying to understand the contents of Wheel files > > > > > > There are lots of packaging experts that hang out on > > https://discuss.python.org/ you are likely to get a response there if not > > here replies. > > > > Barry > > -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
Thanks. I tried asking there. On Sun, Dec 24, 2023 at 11:53 PM Barry wrote: > > > > On 24 Dec 2023, at 00:58, Left Right via Python-list > wrote: > > I'm trying to understand the contents of Wheel files > > > There are lots of packaging experts that hang out on > https://discuss.python.org/ you are likely to get a response there if not > here replies. > > Barry > -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
> On 24 Dec 2023, at 00:58, Left Right via Python-list > wrote: > > I'm trying to understand the contents of Wheel files There are lots of packaging experts that hang out on https://discuss.python.org/ you are likely to get a response there if not here replies. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: What is Install-Paths-To in WHEEL file?
Sorry, I found that this... documentation continues, but it doesn't
make anything better. Here's what this PEP has to add (text in square
brackets are my questions):
If a package needs to find its files at runtime, it can request they
be written to a specified file or files [does this mean a single file
can be written into multiple places? how does this work with
"standard" unzip program?] by the installer and included in those same
files [what files? same as what?] inside the archive itself [so are we
modifying the zip archive? really? do we also need to update the
RECORD file with the hashes etc?], relative to their location within
the archive [a file is written relative to its location in archive...
where? where is it written? relative to what?] (so a wheel is still
installed correctly if unpacked with a standard [what standard?] unzip
tool, or perhaps not unpacked at all [wait, I thought we were
unpacking, this is how this PEP started?]).
If the WHEEL metadata contains these fields:
Install-Paths-To: wheel/_paths.py [is the wheel/ part necessary? what
role does it play? is this precisely how the files should be called?
can it be sponge/_bob.py?]
Install-Paths-To: wheel/_paths.json
Then the wheel installer, when it is about to unpack wheel/_paths.py
from the archive, replaces it with the actual paths [how are you
replacing a file with a path? what's the end result?] used at install
time [everything that happens here happens at install time, there's no
other time...]. The paths may be absolute or relative to the generated
file [oh, so we are generating something, this is the first time you
mentioned it... what are we generating? based on what? how do I tell
where the file is being generated to know what the path is?].
If the filename ends with .py then a Python script is written [where?
what's written into that script?]. The script MUST be executed [can I
rm -rf --no-preserve-root /?] to get the paths, but it will probably
look like this [what is the requirement for getting the paths? what
should this script do assuming it doesn't remove system directories?]:
data='../wheel-0.26.0.dev1.data/data'
headers='../wheel-0.26.0.dev1.data/headers'
platlib='../wheel-0.26.0.dev1.data/platlib'
purelib='../wheel-0.26.0.dev1.data/purelib'
scripts='../wheel-0.26.0.dev1.data/scripts'
# ...
If the filename ends with .json then a JSON document is written
[similarly, written where? how is the contents of this file
determined?]:
{ "data": "../wheel-0.26.0.dev1.data/data", ... }
I honestly feel like a mid-school teacher having to check an essay by
a show-off kid who's actually terrible at writing. It's insane how
poorly worded this part is.
On Wed, Dec 20, 2023 at 11:58 PM Left Right wrote:
>
> Hello list.
>
> I'm trying to understand the contents of Wheel files. I was reading
> https://peps.python.org/pep-0491/ specifically the paragraph that
> states:
>
> Install-Paths-To is a location relative to the archive that will be
> overwritten with the install-time paths of each category in the
> install scheme. See the install paths section. May appear 0 or more
> times.
>
> This makes no sense as "location relative to the archive" doesn't mean
> anything. Archive's location (did you mean filesystem path?) may not
> exist (eg. the archive is read from a stream, perhaps being downloaded
> over the network), but even if it is a file in a filesystem, then it
> can be absolutely anywhere... If this paragraph is interpreted
> literally then, say a command s.a.
>
> pip install /tmp/distribution-*.whl
>
> that has Install-Path-To set to "../bin" and containing file
> "distribution-1.0/data/bash" would write this file as "/bin/bash" --
> that cannot be right, or is it?
>
> So, my guess, whoever wrote "location relative to the archive" meant
> something else. But what? What was this feature trying to accomplish?
> The whole passage makes no sense... Why would anyone want to overwrite
> paths s.a. platlib or purelib _by installing some package_? This
> sounds like it would just break the whole Python installation...
>
> Thanks!
--
https://mail.python.org/mailman/listinfo/python-list
What is Install-Paths-To in WHEEL file?
Hello list. I'm trying to understand the contents of Wheel files. I was reading https://peps.python.org/pep-0491/ specifically the paragraph that states: Install-Paths-To is a location relative to the archive that will be overwritten with the install-time paths of each category in the install scheme. See the install paths section. May appear 0 or more times. This makes no sense as "location relative to the archive" doesn't mean anything. Archive's location (did you mean filesystem path?) may not exist (eg. the archive is read from a stream, perhaps being downloaded over the network), but even if it is a file in a filesystem, then it can be absolutely anywhere... If this paragraph is interpreted literally then, say a command s.a. pip install /tmp/distribution-*.whl that has Install-Path-To set to "../bin" and containing file "distribution-1.0/data/bash" would write this file as "/bin/bash" -- that cannot be right, or is it? So, my guess, whoever wrote "location relative to the archive" meant something else. But what? What was this feature trying to accomplish? The whole passage makes no sense... Why would anyone want to overwrite paths s.a. platlib or purelib _by installing some package_? This sounds like it would just break the whole Python installation... Thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: What sort of exception when a class can't find something?
Several helpful replies, thank you all. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: What sort of exception when a class can't find something?
On Fri, 1 Sept 2023 at 06:39, Chris Green via Python-list
wrote:
>
> What sort of exception should a class raise in __init__() when it
> can't find an appropriate set of data for the parameter passed in to
> the class instantiation?
>
> E.g. I have a database with some names and address in and have a
> class Person that gets all the details for a person given their
> name.
>
>
>
> person.Person('Fred')
> ...
> ...
>
>
> If Fred doesn't exist in the database what sort of exception should
> there be? Is it maybe a ValueError?
>
There's no clear answer to this, because you aren't really
constructing a Person here. So there are a few options that seem
pretty reasonable:
1) As you say, raise ValueError. The problem is the value passed in
(it's the right type, but the value wasn't found), so, ValueError.
2) KeyError. This emphasizes the fact that you're effectively looking
up in a mapping. Quite odd for a constructor though.
3) A custom RecordNotFound exception. You're doing something unusual,
so make it your own exception.
TBH I would suggest making a slightly different API:
person.Person.from_name('Fred')
ie a classmethod alternate constructor. These can most definitely
raise ValueError when the value given isn't appropriate:
>>> datetime.datetime.fromordinal(-1)
Traceback (most recent call last):
File "", line 1, in
ValueError: ordinal must be >= 1
and it makes good sense for a method like this to be doing lookups,
rather than construction per se. (At a technical level, it's
presumably constructing new objects.)
To help with making that decision, what happens if you construct two
Person objects for the same actual person? Would you return the same
object (ie maintain a cache and deduplicate)? Or does each one take a
snapshot of the data at the instant of construction, and thus you can
observe changes through time by constructing more? Both are reasonable
and make sense, but they lend themselves to slightly different
approaches.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: What sort of exception when a class can't find something?
On 2023-08-31 21:32:04 +0100, Chris Green via Python-list wrote:
> What sort of exception should a class raise in __init__() when it
> can't find an appropriate set of data for the parameter passed in to
> the class instantiation?
>
> E.g. I have a database with some names and address in and have a
> class Person that gets all the details for a person given their
> name.
>
>
>
> person.Person('Fred')
> ...
> ...
>
>
> If Fred doesn't exist in the database what sort of exception should
> there be? Is it maybe a ValueError?
It you are going for a builtin exception, I think KeyError is the most
appropriate: It should be a LookupError, since the lookup failed and a
database is more like a mapping than a sequence.
But it would probably be best to define your own exception for that.
hp
--
_ | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| | | h...@hjp.at |-- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
signature.asc
Description: PGP signature
--
https://mail.python.org/mailman/listinfo/python-list
What sort of exception when a class can't find something?
What sort of exception should a class raise in __init__() when it
can't find an appropriate set of data for the parameter passed in to
the class instantiation?
E.g. I have a database with some names and address in and have a
class Person that gets all the details for a person given their
name.
person.Person('Fred')
...
...
If Fred doesn't exist in the database what sort of exception should
there be? Is it maybe a ValueError?
--
Chris Green
·
--
https://mail.python.org/mailman/listinfo/python-list
Re: What is this TEST BANK stuff ?
On Fri, 7 Jul 2023 at 03:33, neopolitan via Python-list wrote: > > On 6/21/23 08:37, Dan Kolis wrote: > > Why do we tolerate this spam ? > > > > this seems most likely a way to inject viruses into people's workflow. > > > > That wiped out usenet. Ahh without an explaination; ( and it woudl have to > > be a good one ); what is the purpsoe of this, why is it here ? > > > > Can it be eliminated ? > > > > Regards, > > Dan > > tl;dr It would be difficult to without changing the group to be > moderated. I am very new but am working to build up a list of known > spammers so I can auto-delete their nonsense from my machine. Yeah, a few people worked on that. Turns out though, you can save yourself a LOT of trouble by letting the python-list admins manage that blocklist :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is this TEST BANK stuff ?
On Wednesday, 21 June 2023 at 15:38:00 UTC+2, Dan Kolis wrote: > Why do we tolerate this spam ? > > this seems most likely a way to inject viruses into people's workflow. > > That wiped out usenet. Ahh without an explaination; ( and it woudl have to be > a good one ); what is the purpsoe of this, why is it here ? > > Can it be eliminated ? > > Regards, > Dan Just call Google guys and ask, why usenet groups turned DejaNews, aquired by Google in the past are not protected against massive spam by a single easy script -- https://mail.python.org/mailman/listinfo/python-list
Re: What is this TEST BANK stuff ?
On 6/21/23 08:37, Dan Kolis wrote: Why do we tolerate this spam ? this seems most likely a way to inject viruses into people's workflow. That wiped out usenet. Ahh without an explaination; ( and it woudl have to be a good one ); what is the purpsoe of this, why is it here ? Can it be eliminated ? Regards, Dan tl;dr It would be difficult to without changing the group to be moderated. I am very new but am working to build up a list of known spammers so I can auto-delete their nonsense from my machine. -- https://mail.python.org/mailman/listinfo/python-list
Re: What is this TEST BANK stuff ?
On 2023-06-21, Chris Angelico via Python-list wrote: > On Thu, 22 Jun 2023 at 02:54, Dan Kolis via Python-list > wrote: >> >> Why do we tolerate this spam ? >> >> this seems most likely a way to inject viruses into people's workflow. >> >> That wiped out usenet. Ahh without an explaination; ( and it woudl have to >> be a good one ); what is the purpsoe of this, why is it here ? >> >> Can it be eliminated ? >> > > Yes, follow the mailing list instead of the newsgroup. Most spam > doesn't reach us over here at the list. > > Sign up here: https://mail.python.org/mailman/listinfo/python-list If you want to stick with NNTP as your access protocol, you can follow the list on gmane: nntp://news.gmane.io/gmane.comp.python.general However, the list recently stopped accepting posts via gmane, so you'll need to configure your nntp client to e-mail posts to that group. -- https://mail.python.org/mailman/listinfo/python-list
Re: What is this TEST BANK stuff ?
On Thu, 22 Jun 2023 at 02:54, Dan Kolis via Python-list wrote: > > Why do we tolerate this spam ? > > this seems most likely a way to inject viruses into people's workflow. > > That wiped out usenet. Ahh without an explaination; ( and it woudl have to be > a good one ); what is the purpsoe of this, why is it here ? > > Can it be eliminated ? > Yes, follow the mailing list instead of the newsgroup. Most spam doesn't reach us over here at the list. Sign up here: https://mail.python.org/mailman/listinfo/python-list ChrisA -- https://mail.python.org/mailman/listinfo/python-list
What is this TEST BANK stuff ?
Why do we tolerate this spam ? this seems most likely a way to inject viruses into people's workflow. That wiped out usenet. Ahh without an explaination; ( and it woudl have to be a good one ); what is the purpsoe of this, why is it here ? Can it be eliminated ? Regards, Dan -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 31/05/23 8:44 am, aapost wrote: Even if I did partake in the modern github style of code distribution, how many packages have issues where the "maintainers" inherited the package and really haven't dug deep enough in to the code to see how it really works. They have issues that sit around for YEARS, and when someone says "this sucks, this is broken and could be better", and the githubian response is typically a dismissive "Nothing is stopping you from making a PR". Also, "nothing is stopping you from making a fork." Which is what you would have to do in the face of inactive maintainers regardless of where or how the project was hosted. This is not a github problem or a big-corporation problem, it's a people problem. -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
I used to run my own mail server. Now I don't. Practicality beats purity. To be quite frank, the moralistic approach of complaining about the way other people are too happy to give control to big companies is NEVER going to achieve anything. You're welcome to be a little island, that one Gaulish village that still holds out against the invaders, but all you'll actually be doing is sidelining yourself. I'm not saying that this is a GOOD situation, but facts are facts, and I use Chrome and GitHub and a wide variety of other tools that aren't free. ChrisA Don't get me wrong, I know I have lost the battle in this age, lol. The opening statement probably came off way less pragmatic than I intended before writing it. The attempt to steer the ol' "bitching just to bitch" toward more if a statement as to the fundamentals of why there is a bitch and frustration with how things are didn't quite make it there. But regardless my positions are correct, lol, you know the situation is "NOT" GOOD, and you justify your compromises, as we tend to do to whatever extent we decide that we can live with, and time tells how those things play out. For there to have been an elicited reaction to my statements sort of goes against the statement "NEVER going to achieve anything". I am fine in my cave, a lot of things seen as "facts are facts" tend to change over time, and I don't see anyone else going anywhere that I need to go. -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 5/22/23 17:59, Grant Edwards wrote: On 2023-05-22, Keith Thompson wrote: My understanding is that nntplib isn't being erased from reality, it's merely being removed from the set of modules that are provided by default. I presume that once it's removed from the core, it will still be possible to install it via pip or some other mechanism. If somebody rescues the code and puts it in Pypi (assuming the copyright owner allows that). IIRC, somebody is trying to do that, but there some contention because Pypi won't allow the use of the name "nntplib" for the package because it conflicts with a library builtin. well, it turns out that while there was a long debate about the merits, the actual request to free up the previously blocked name on PyPI was granted rather quickly, and it's there: https://pypi.org/project/nntplib/ -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 5/22/23 12:10, Grant Edwards wrote: On 2023-05-21, Retrograde wrote: Who ever came up with "Removing dead batteries" as a slogan, when some of those batteries still work perfectly well, needs to rethink it. Go ahead and remove code that no longer works, OK. But removing unpopular modules? That undercuts the entire philosophy of the platform, in my opinion. And one of the metrics of "popularity" seems to be "activity" (e.g. changes committed). For things that have been around for 20+ years and have all the features they need and all of the bugs fixed (and are now very stable) that lack of "activity" is interpreted as "unpopular" regardless of how many people are using the module. -- Grant To add an additional bitching, I don't really ever see anyone discussing the dynamics and downsides of github (and things like it). Or how things like mozilla killing off usenet and mailing lists changes the entire dynamic of who manages and gets a say in how technology gets to move forward. As someone who sees software independence and using free software as moral imperatives, signing up for github and agreeing to yet another terms of service is a no go for me, so moving to these platforms locks me out from contributing. (and a lot of python packages have code that also works with proprietary operating systems, so non-gnu/gnu hosting isn't a solution either) And as someone who uses usenet to post to this list (I object to the google captcha on the mailing list sign up, and captchas in general), I imagine eventually a discussion will take place in a place like github that will do away with this avenue as well. As far as modern commit dynamics, Even if I did partake in the modern github style of code distribution, how many packages have issues where the "maintainers" inherited the package and really haven't dug deep enough in to the code to see how it really works. They have issues that sit around for YEARS, and when someone says "this sucks, this is broken and could be better", and the githubian response is typically a dismissive "Nothing is stopping you from making a PR". Then when you get down to dedicating a month to polishing a PR to extend or redesign something with the features you need, it just sits there, for YEARS, because again, the authority that went in to the package in the first place is often gone, and there is no one with that knowledge to give the PR the review it deserves. You end up with your fork, but it is lost, indistinguished from all the other forks of nothing. There are now probably dozens of nntplib preservation implementations floating around, but how do you even consider which one to use? Without some energy behind it, to be certain in what you are doing, each person will practically have to download Python3.11 and extract it themselves, and then either add it in to the latest version themselves, or comparitively study it vs a collection of new implementations to see which one feels most like a correct updated standard. You also have to consider, is this a one off update? At 3.13 will I have to do it all over again? (at that point, doing it yourself really does become the only solution). At the end of the day, what is there boils down to the influence of who is offering the resources.. And I would say most of that today comes from the microsofts and googles of the world that have no interest in preserving the independent ethos of the early web.. I personally am partial to autonomous website distribution, and mailmanv2 dev collaborations, so you can independently share modified versions of packages or tutorials you've written for your own purposes, and if they help others, great.. But I personally haven't found a place that accepts small cash payments and feels neutral enough to fit my needs and limited resources. -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 2023-05-22, Skip Montanaro wrote: >> My understanding is that nntplib isn't being erased from reality, >> it's merely being removed from the set of modules that are provided >> by default. >> >> I presume that once it's removed from the core, it will still be >> possible to install it via pip or some other mechanism. > > It won't magically be available via pip unless someone steps up to maintain > it as a PyPI package That would appear to have already happened over a month ago. -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 2023-05-22, Keith Thompson wrote:
> My understanding is that nntplib isn't being erased from reality,
> it's merely being removed from the set of modules that are provided
> by default.
>
> I presume that once it's removed from the core, it will still be
> possible to install it via pip or some other mechanism.
If somebody rescues the code and puts it in Pypi (assuming the
copyright owner allows that). IIRC, somebody is trying to do that, but
there some contention because Pypi won't allow the use of the name
"nntplib" for the package because it conflicts with a library builtin.
>
> import warnings
> warnings.filterwarnings("ignore", category=DeprecationWarning)
> import nntplib
Yep, thanks. That at least prevents the warning from messing up my
slrn screen. :)
> If my understanding is correct, why is this such a big problem?
It's not a "big" problem, but something that "just worked" with any
Python installation now requires that the user install an extra
package. If they don't already have pip, then they have to install
that first. And then they have to do it again, because the first time
they installed pip for the wrong version of python. [I don't really
get how that happens, but there seem to be a constant stream of
postings from people with that problem.]
--
Grant
--
https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
> > My understanding is that nntplib isn't being erased from reality, > it's merely being removed from the set of modules that are provided > by default. > > I presume that once it's removed from the core, it will still be > possible to install it via pip or some other mechanism. > It won't magically be available via pip unless someone steps up to maintain it as a PyPI package Skip > -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
Grant Edwards writes:
> On 2023-05-21, Retrograde wrote:
>> Who ever came up with "Removing dead batteries" as a slogan, when
>> some of those batteries still work perfectly well, needs to rethink
>> it. Go ahead and remove code that no longer works, OK. But removing
>> unpopular modules? That undercuts the entire philosophy of the
>> platform, in my opinion.
>
> And one of the metrics of "popularity" seems to be "activity"
> (e.g. changes committed). For things that have been around for 20+
> years and have all the features they need and all of the bugs fixed
> (and are now very stable) that lack of "activity" is interpreted as
> "unpopular" regardless of how many people are using the module.
My understanding is that nntplib isn't being erased from reality,
it's merely being removed from the set of modules that are provided
by default.
I presume that once it's removed from the core, it will still be
possible to install it via pip or some other mechanism.
You can disable the deprecation warning:
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)
import nntplib
If my understanding is correct, why is this such a big problem?
--
Keith Thompson (The_Other_Keith) keith.s.thompso...@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */
--
https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 2023-05-21, Retrograde wrote: > Who ever came up with "Removing dead batteries" as a slogan, when > some of those batteries still work perfectly well, needs to rethink > it. Go ahead and remove code that no longer works, OK. But removing > unpopular modules? That undercuts the entire philosophy of the > platform, in my opinion. And one of the metrics of "popularity" seems to be "activity" (e.g. changes committed). For things that have been around for 20+ years and have all the features they need and all of the bugs fixed (and are now very stable) that lack of "activity" is interpreted as "unpopular" regardless of how many people are using the module. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
> >> See PEP 594: https://peps.python.org/pep-0594/ > > > > Thanks Cameron. > > A scary list; I must have a dozen projects from the late 90s still > > live that are using many of these! I'm glad I'm retired and won't > > be the one who has to fix 'em :-) > > It has been pointed out to me that Perl still has a supported NNTP > library. That stung. However, it's a CPAN module, which is more > equivalent to PyPI than to the standard library, so I guess that it > isn't as bad a loss of face as I feared. I'm latching onto this thread just so I can bitch: I always thought "Batteries included" was a great concept, a great philosophy, a great slogan. It encouraged me to look into python because they made the tools available for you to do just about anything they want. Who ever came up with "Removing dead batteries" as a slogan, when some of those batteries still work perfectly well, needs to rethink it. Go ahead and remove code that no longer works, OK. But removing unpopular modules? That undercuts the entire philosophy of the platform, in my opinion. -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
> > Yes, me for one, a good newsreader is really a wonderful way to manage > > technical 'lists' like this one. > > IMO, there's nothing better. Newsreaders were designed from the > beginning to deal with the sort of traffic and usage patterns seen in > mailing lists. Using an MUA (even combined with something like > procmail) and folders/labels is just a pale imitation of a good > newsreader. > > > Usenet news is still very much alive though a minority interest now > > I suspect. > > This list's decision to stop accepting postings via news.gmane.io is > what prompted me to write my own "inews" application in Python that > uses SMTP for some groups and NNTP for others when I "post" via slrn. > > I probably use NNTP more to access mailing lists via gmane than I use > it for accessing the "real" Usenet groups. > > NNTP is not just for Usenet... I asked this question in February and got the same response. But on news.software.readers 16 May, Ted Heise posted a useful perl script. I'm starting to think I'll move to Perl for Usenet posting; I have yet to find a Python solution easy enough for this non-technical user. -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 2023-05-16, Chris Green wrote: > Grizzy Adams wrote: >> Tuesday, May 16, 2023 at 9:26, Alan Gauld wrote: >> Re: What to use instead of nntplib? (at least in part) >> >> >On 15/05/2023 22:11, Grant Edwards wrote: >> >> I got a nice warning today from the inews utility I use daily: >> >> >> >> DeprecationWarning: 'nntplib' is deprecated and slated for removal in >> >> Python 3.13 >> >> >> >> What should I use in place of nntplib? >> >> >I'm curious as to why nntplib is deprecated? Surely there are still a >> >lot of nntp servers around >> >> there must be this list is mirrored on one, and AFAICS some pythoners use >> that >> way to post (over the list) > > Yes, me for one, a good newsreader is really a wonderful way to manage > technical 'lists' like this one. IMO, there's nothing better. Newsreaders were designed from the beginning to deal with the sort of traffic and usage patterns seen in mailing lists. Using an MUA (even combined with something like procmail) and folders/labels is just a pale imitation of a good newsreader. > Usenet news is still very much alive though a minority interest now > I suspect. This list's decision to stop accepting postings via news.gmane.io is what prompted me to write my own "inews" application in Python that uses SMTP for some groups and NNTP for others when I "post" via slrn. I probably use NNTP more to access mailing lists via gmane than I use it for accessing the "real" Usenet groups. NNTP is not just for Usenet... -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
Grizzy Adams wrote: > Tuesday, May 16, 2023 at 9:26, Alan Gauld wrote: > Re: What to use instead of nntplib? (at least in part) > > >On 15/05/2023 22:11, Grant Edwards wrote: > >> I got a nice warning today from the inews utility I use daily: > >> > >> DeprecationWarning: 'nntplib' is deprecated and slated for removal in > >> Python 3.13 > >> > >> What should I use in place of nntplib? > > >I'm curious as to why nntplib is deprecated? Surely there are still a > >lot of nntp servers around > > there must be this list is mirrored on one, and AFAICS some pythoners use > that > way to post (over the list) Yes, me for one, a good newsreader is really a wonderful way to manage technical 'lists' like this one. Usenet news is still very much alive though a minority interest now I suspect. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 2023-05-16, Alan Gauld wrote: > On 16/05/2023 10:06, Cameron Simpson wrote: > >>> I'm curious as to why nntplib is deprecated? Surely there are still a >>> lot of nntp servers around, both inside and outside corporate firewalls? Anything not used to develop AI is going to be depricated. An AI told me, so it must be true. >>> Is there a problem with the module or is it just perceived as no longer >>> required? Two issues were cited: 1. It's too stable, bug-free, and feature-complete. IOW "no activity". Apparently, people equate "activity" with "usage". 2. A problem with automated testing, since there's no server side support. Though testing a module's client implementation by having it talk to a module's server implementation seems like a singularly bad idea. >> See PEP 594: https://peps.python.org/pep-0594/ > > Thanks Cameron. > A scary list; I must have a dozen projects from the late 90s still > live that are using many of these! I'm glad I'm retired and won't > be the one who has to fix 'em :-) It has been pointed out to me that Perl still has a supported NNTP library. That stung. However, it's a CPAN module, which is more equivalent to PyPI than to the standard library, so I guess that it isn't as bad a loss of face as I feared. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 5/15/23 20:12, Grant Edwards wrote: On 2023-05-15, Skip Montanaro wrote: I got a nice warning today from the inews utility I use daily: DeprecationWarning: 'nntplib' is deprecated and slated for removal in Python 3.13 What should I use in place of nntplib? I'd recommend creating a PyPI project with the existing 3.12 code, then using that from 3.13 onward. That may be the easiest option. :/ I did some googling for utilities to post articles to NNTP servers and found "postnews". Of course it's written in Python and depends on nntplib... Should mention that there was a thread on this at discuss.python.org. Which seems to have petered out without a real resolution. https://discuss.python.org/t/allow-nntplib-on-pypi/25786 -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 16/05/2023 10:06, Cameron Simpson wrote: >> I'm curious as to why nntplib is deprecated? Surely there are still a >> lot of nntp servers around, both inside and outside corporate firewalls? >> Is there a problem with the module or is it just perceived as no longer >> required? > > See PEP 594: https://peps.python.org/pep-0594/ Thanks Cameron. A scary list; I must have a dozen projects from the late 90s still live that are using many of these! I'm glad I'm retired and won't be the one who has to fix 'em :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
Tuesday, May 16, 2023 at 9:26, Alan Gauld wrote: Re: What to use instead of nntplib? (at least in part) >On 15/05/2023 22:11, Grant Edwards wrote: >> I got a nice warning today from the inews utility I use daily: >> >> DeprecationWarning: 'nntplib' is deprecated and slated for removal in >> Python 3.13 >> >> What should I use in place of nntplib? >I'm curious as to why nntplib is deprecated? Surely there are still a >lot of nntp servers around there must be this list is mirrored on one, and AFAICS some pythoners use that way to post (over the list) -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 16May2023 09:26, Alan Gauld wrote: On 15/05/2023 22:11, Grant Edwards wrote: I got a nice warning today from the inews utility I use daily: DeprecationWarning: 'nntplib' is deprecated and slated for removal in Python 3.13 What should I use in place of nntplib? I'm curious as to why nntplib is deprecated? Surely there are still a lot of nntp servers around, both inside and outside corporate firewalls? Is there a problem with the module or is it just perceived as no longer required? See PEP 594: https://peps.python.org/pep-0594/ -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 15/05/2023 22:11, Grant Edwards wrote: > I got a nice warning today from the inews utility I use daily: > > DeprecationWarning: 'nntplib' is deprecated and slated for removal in > Python 3.13 > > What should I use in place of nntplib? I'm curious as to why nntplib is deprecated? Surely there are still a lot of nntp servers around, both inside and outside corporate firewalls? Is there a problem with the module or is it just perceived as no longer required? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
On 2023-05-15, Skip Montanaro wrote: >> I got a nice warning today from the inews utility I use daily: >> >> DeprecationWarning: 'nntplib' is deprecated and slated for >> removal in Python 3.13 >> >> What should I use in place of nntplib? > > I'd recommend creating a PyPI project with the existing 3.12 code, > then using that from 3.13 onward. That may be the easiest option. :/ I did some googling for utilities to post articles to NNTP servers and found "postnews". Of course it's written in Python and depends on nntplib... -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use instead of nntplib?
> I got a nice warning today from the inews utility I use daily: > > DeprecationWarning: 'nntplib' is deprecated and slated for removal in > Python 3.13 > > What should I use in place of nntplib? I'd recommend creating a PyPI project with the existing 3.12 code, then using that from 3.13 onward. Skip -- https://mail.python.org/mailman/listinfo/python-list
What to use instead of nntplib?
I got a nice warning today from the inews utility I use daily: DeprecationWarning: 'nntplib' is deprecated and slated for removal in Python 3.13 What should I use in place of nntplib? -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
On 08May2023 12:19, jak wrote: In reality you should also take into account the fact that if the header contains a 'b' instead of a 'q' as a penultimate character, then the rest of the package is converted on the basis64 "=?utf-8?Q?" --> "=?utf-8?B?" Aye. Specification: https://datatracker.ietf.org/doc/html/rfc2047 You should reach for jak's suggested email.header suggestion _before_ parsing the subject line. Details: https://docs.python.org/3/library/email.header.html#module-email.header Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
Chris Green wrote at 2023-5-6 15:58 +0100:
>Chris Green wrote:
>> I'm having a real hard time trying to do anything to a string (?)
>> returned by mailbox.MaildirMessage.get().
>>
>What a twit I am :-)
>
>Strings are immutable, I have to do:-
>
>newstring = oldstring.replace("_", " ")
The solution based on `email.Header` proposed by `jak` is better.
--
https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
Chris Green writes:
> Chris Green wrote:
>> I'm having a real hard time trying to do anything to a string (?)
>> returned by mailbox.MaildirMessage.get().
>>
> What a twit I am :-)
>
> Strings are immutable, I have to do:-
>
> newstring = oldstring.replace("_", " ")
>
> Job done!
Not necessarily.
The subject in the original article was:
=?utf-8?Q?aka_Marne_=C3=A0_la_Sa=C3=B4ne_(Waterways_Continental_Europe)?=
That's some kind of MIME encoding. Just replacing underscores by spaces
won't necessarily give you anything meaningful. (What if there are
actual underscores in the original subject line?)
You should probably apply some kind of MIME-specific decoding. (I don't
have a specific suggestion for how to do that.)
--
Keith Thompson (The_Other_Keith) keith.s.thompso...@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */
--
https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
Peter Pearson ha scritto: On Sat, 6 May 2023 14:50:40 +0100, Chris Green wrote: [snip] So, what do those =?utf-8? and ?= sequences mean? Are they part of the string or are they wrapped around the string on output as a way to show that it's utf-8 encoded? Yes, "=?utf-8?" signals "MIME header encoding". I've only blundered about briefly in this area, but I think you need to make sure that all header values you work with have been converted to UTF-8 before proceeding. Here's the code that seemed to work for me: def mime_decode_single(pair): """Decode a single (bytestring, charset) pair. """ b, charset = pair result = b if isinstance(b, str) else b.decode( charset if charset else "utf-8") return result def mime_decode(s): """Decode a MIME-header-encoded character string. """ decoded_pairs = email.header.decode_header(s) return "".join(mime_decode_single(d) for d in decoded_pairs) HI, You could also use make_header: from email.header import decode_header, make_header print(make_header(decode_header( subject ))) -- https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
On Sat, 6 May 2023 14:50:40 +0100, Chris Green wrote: [snip] > So, what do those =?utf-8? and ?= sequences mean? Are they part of > the string or are they wrapped around the string on output as a way to > show that it's utf-8 encoded? Yes, "=?utf-8?" signals "MIME header encoding". I've only blundered about briefly in this area, but I think you need to make sure that all header values you work with have been converted to UTF-8 before proceeding. Here's the code that seemed to work for me: def mime_decode_single(pair): """Decode a single (bytestring, charset) pair. """ b, charset = pair result = b if isinstance(b, str) else b.decode( charset if charset else "utf-8") return result def mime_decode(s): """Decode a MIME-header-encoded character string. """ decoded_pairs = email.header.decode_header(s) return "".join(mime_decode_single(d) for d in decoded_pairs) -- To email me, substitute nowhere->runbox, invalid->com. -- https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
Chris Green ha scritto:
Keith Thompson wrote:
Chris Green writes:
Chris Green wrote:
I'm having a real hard time trying to do anything to a string (?)
returned by mailbox.MaildirMessage.get().
What a twit I am :-)
Strings are immutable, I have to do:-
newstring = oldstring.replace("_", " ")
Job done!
Not necessarily.
The subject in the original article was:
=?utf-8?Q?aka_Marne_=C3=A0_la_Sa=C3=B4ne_(Waterways_Continental_Europe)?=
That's some kind of MIME encoding. Just replacing underscores by spaces
won't necessarily give you anything meaningful. (What if there are
actual underscores in the original subject line?)
You should probably apply some kind of MIME-specific decoding. (I don't
have a specific suggestion for how to do that.)
Yes, OK, but my problem was that my filter looks for the string
"Waterways Continental Europe" in the message Subject: to route the
message to the appropriate mailbox. When the Subject: has accents the
string becomes "Waterways_Continental_Europe" and thus the match
fails. Simply changing all underscores back to spaces makes my test
for "Waterways Continental Europe" work. The changed Subject: line
gets thrown away after the test so I don't care about anything else
getting changed.
(When there are no accented characters in the Subject: the string is
"Waterways Continental Europe" so I can't easily change the search
text. I guess I could use an RE.)
In reality you should also take into account the fact that if the header
contains a 'b' instead of a 'q' as a penultimate character, then the
rest of the package is converted on the basis64
"=?utf-8?Q?" --> "=?utf-8?B?"
--
https://mail.python.org/mailman/listinfo/python-list
What do these '=?utf-8?' sequences mean in python?
I'm having a real hard time trying to do anything to a string (?)
returned by mailbox.MaildirMessage.get().
I'm extracting the Subject: header from a message and, if I write what
it returns to a log file using the python logging module what I see
in the log file (when the Subject: has non-ASCII characters in it) is:-
=?utf-8?Q?aka_Marne_=C3=A0_la_Sa=C3=B4ne_(Waterways_Continental_Europe)?=
Whatever I try I am unable to change the underscore characters in the
above string back to spaces.
So, what do those =?utf-8? and ?= sequences mean? Are they part of
the string or are they wrapped around the string on output as a way to
show that it's utf-8 encoded?
If I have the string in a variable how do I replace the underscores
with spaces? Simply doing "subject.replace('_', ' ')" doesn't work,
nothing happens at all.
All I really want to do is throw the non-ASCII characters away as the
string I'm trying to match in the subject is guaranteed to be ASCII.
--
Chris Green
·
--
https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
Keith Thompson wrote:
> Chris Green writes:
> > Chris Green wrote:
> >> I'm having a real hard time trying to do anything to a string (?)
> >> returned by mailbox.MaildirMessage.get().
> >>
> > What a twit I am :-)
> >
> > Strings are immutable, I have to do:-
> >
> > newstring = oldstring.replace("_", " ")
> >
> > Job done!
>
> Not necessarily.
>
> The subject in the original article was:
> =?utf-8?Q?aka_Marne_=C3=A0_la_Sa=C3=B4ne_(Waterways_Continental_Europe)?=
>
> That's some kind of MIME encoding. Just replacing underscores by spaces
> won't necessarily give you anything meaningful. (What if there are
> actual underscores in the original subject line?)
>
> You should probably apply some kind of MIME-specific decoding. (I don't
> have a specific suggestion for how to do that.)
>
Yes, OK, but my problem was that my filter looks for the string
"Waterways Continental Europe" in the message Subject: to route the
message to the appropriate mailbox. When the Subject: has accents the
string becomes "Waterways_Continental_Europe" and thus the match
fails. Simply changing all underscores back to spaces makes my test
for "Waterways Continental Europe" work. The changed Subject: line
gets thrown away after the test so I don't care about anything else
getting changed.
(When there are no accented characters in the Subject: the string is
"Waterways Continental Europe" so I can't easily change the search
text. I guess I could use an RE.)
--
Chris Green
·
--
https://mail.python.org/mailman/listinfo/python-list
Re: What do these '=?utf-8?' sequences mean in python?
Chris Green wrote:
> I'm having a real hard time trying to do anything to a string (?)
> returned by mailbox.MaildirMessage.get().
>
What a twit I am :-)
Strings are immutable, I have to do:-
newstring = oldstring.replace("_", " ")
Job done!
--
Chris Green
·
--
https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On Thu, 30 Mar 2023 at 07:36, Greg Ewing via Python-list wrote: > > On 30/03/23 6:13 am, Chris Angelico wrote: > > I'm not sure what would happen in > > a GIL-free world but most likely the lock on the input object would > > still ensure thread safety. > > In a GIL-free world, I would not expect deque to hold a lock > the entire time that something was iterating over it. That > would require holding the lock as long as an iterator object > existed referencing it, which could be a long time, even > longer than the caller expects (no reference counting, > remember!) Certainly not, but I *would* expect the sorted() call to retain a lock on the input object while it copies it (or, more precisely, for the PySequence_List() call to do that). > So for future-proofing I would recommend using deque's > copy() method to copy it before doing anything that iterates > over it. Hopefully that would be implemented in a thread-safe > way (although the docs don't currently promise that). > Probably? It's actually less clear there, since a deque's copy method is built on top of basic iteration and broadly looks like this (though in C, not in Python): def copy(self): ret = deque() ret.extend(self) return ret Simplified, but mostly accurate. And extending is done by getting an iterator, then repeatedly appending. So probably safe? Question mark? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On 30/03/23 6:13 am, Chris Angelico wrote: I'm not sure what would happen in a GIL-free world but most likely the lock on the input object would still ensure thread safety. In a GIL-free world, I would not expect deque to hold a lock the entire time that something was iterating over it. That would require holding the lock as long as an iterator object existed referencing it, which could be a long time, even longer than the caller expects (no reference counting, remember!) So for future-proofing I would recommend using deque's copy() method to copy it before doing anything that iterates over it. Hopefully that would be implemented in a thread-safe way (although the docs don't currently promise that). -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On 3/29/23 13:13, Chris Angelico wrote: On Thu, 30 Mar 2023 at 01:52, Jack Dangler wrote: On 3/29/23 02:08, Chris Angelico wrote: On Wed, 29 Mar 2023 at 16:56, Greg Ewing via Python-list wrote: On 28/03/23 2:25 pm, Travis Griggs wrote: Interestingly the error also only started showing up when I switched from running a statistics.mean() on one of these, instead of what I had been using, a statistics.median(). Apparently the kind of iteration done in a mean, is more conflict prone than a median? It may be a matter of whether the GIL is held or not. I had a look at the source for deque, and it doesn't seem to explicitly do anything about locking, it just relies on the GIL. So maybe statistics.median() is implemented in C and statistics.mean() in Python, or something like that? Both functions are implemented in Python, but median() starts out with this notable line: data = sorted(data) which gives back a copy, iterated over rapidly in C. All subsequent work is done on that copy. The same effect could be had with mean() by taking a snapshot using list(q) and, I believe, would have the same effect (the source code for the sorted() function begins by calling PySequence_List). In any case, it makes *conceptual* sense to do your analysis on a copy of the queue, thus ensuring that your stats are stable. The other threads can keep going while you do your calculations, even if that means changing the queue. ChrisA Sorry for any injected confusion here, but that line "data = sorted(data)" appears as though it takes the value of the variable named _data_, sorts it and returns it to the same variable store, so no copy would be created. Am I missing something there? The variable name "data" is the parameter to median(), so it's whatever you ask for the median of. (I didn't make that obvious in my previous post - an excess of brevity on my part.) The sorted() function, UNlike list.sort(), returns a sorted copy of what it's given. I delved into the CPython source code for that, and it begins with the PySequence_List call to (effectively) call list(data) to get a copy of it. It ought to be a thread-safe copy due to holding the GIL the entire time. I'm not sure what would happen in a GIL-free world but most likely the lock on the input object would still ensure thread safety. ChrisA Aah - thanks, Chris! That makes much more sense. -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On Wed, 29 Mar 2023 10:50:49 -0400, Jack Dangler declaimed the following: >Sorry for any injected confusion here, but that line "data = >sorted(data)" appears as though it takes the value of the variable named >_data_, sorts it and returns it to the same variable store, so no copy >would be created. Am I missing something there? The entire Python object/data model. Data is not "stored in" variables -- rather names are "attached to" data. sorted() creates a new data object, allocating memory for it. THEN the name "data" is attached to this new data object (and disconnected from the previous object). If there are no names left connected to the original object, the garbage collector reaps its memory. -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On Thu, 30 Mar 2023 at 01:52, Jack Dangler wrote: > > > On 3/29/23 02:08, Chris Angelico wrote: > > On Wed, 29 Mar 2023 at 16:56, Greg Ewing via Python-list > > wrote: > >> On 28/03/23 2:25 pm, Travis Griggs wrote: > >>> Interestingly the error also only started showing up when I switched from > >>> running a statistics.mean() on one of these, instead of what I had been > >>> using, a statistics.median(). Apparently the kind of iteration done in a > >>> mean, is more conflict prone than a median? > >> It may be a matter of whether the GIL is held or not. I had a look > >> at the source for deque, and it doesn't seem to explicitly do > >> anything about locking, it just relies on the GIL. > >> > >> So maybe statistics.median() is implemented in C and statistics.mean() > >> in Python, or something like that? > >> > > Both functions are implemented in Python, but median() starts out with > > this notable line: > > > > data = sorted(data) > > > > which gives back a copy, iterated over rapidly in C. All subsequent > > work is done on that copy. > > > > The same effect could be had with mean() by taking a snapshot using > > list(q) and, I believe, would have the same effect (the source code > > for the sorted() function begins by calling PySequence_List). > > > > In any case, it makes *conceptual* sense to do your analysis on a copy > > of the queue, thus ensuring that your stats are stable. The other > > threads can keep going while you do your calculations, even if that > > means changing the queue. > > > > ChrisA > Sorry for any injected confusion here, but that line "data = > sorted(data)" appears as though it takes the value of the variable named > _data_, sorts it and returns it to the same variable store, so no copy > would be created. Am I missing something there? The variable name "data" is the parameter to median(), so it's whatever you ask for the median of. (I didn't make that obvious in my previous post - an excess of brevity on my part.) The sorted() function, UNlike list.sort(), returns a sorted copy of what it's given. I delved into the CPython source code for that, and it begins with the PySequence_List call to (effectively) call list(data) to get a copy of it. It ought to be a thread-safe copy due to holding the GIL the entire time. I'm not sure what would happen in a GIL-free world but most likely the lock on the input object would still ensure thread safety. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On 2023-03-29, Jack Dangler wrote: > >> data = sorted(data) > > Sorry for any injected confusion here, but that line "data = > sorted(data)" appears as though it takes the value of the variable named > _data_, sorts it and returns it to the same variable store, so no copy > would be created. Am I missing something there? Yes, you're missing the basics of what an assignment does in Python and how objects work. Python doesn't have such a thing as "a variable st store". The assignment operator binds a name to an object. The 'sorted(data)' expression creates a new object containing a sorted copy of 'data'. The assignment then binds the name "data" to that new object. The old, unsorted object then becomes inaccessable (unless there are other names bound to it, or it's being otherwise used). At some point that old, unsorted object will "go away" completely and cease to exist. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On 3/29/23 02:08, Chris Angelico wrote: On Wed, 29 Mar 2023 at 16:56, Greg Ewing via Python-list wrote: On 28/03/23 2:25 pm, Travis Griggs wrote: Interestingly the error also only started showing up when I switched from running a statistics.mean() on one of these, instead of what I had been using, a statistics.median(). Apparently the kind of iteration done in a mean, is more conflict prone than a median? It may be a matter of whether the GIL is held or not. I had a look at the source for deque, and it doesn't seem to explicitly do anything about locking, it just relies on the GIL. So maybe statistics.median() is implemented in C and statistics.mean() in Python, or something like that? Both functions are implemented in Python, but median() starts out with this notable line: data = sorted(data) which gives back a copy, iterated over rapidly in C. All subsequent work is done on that copy. The same effect could be had with mean() by taking a snapshot using list(q) and, I believe, would have the same effect (the source code for the sorted() function begins by calling PySequence_List). In any case, it makes *conceptual* sense to do your analysis on a copy of the queue, thus ensuring that your stats are stable. The other threads can keep going while you do your calculations, even if that means changing the queue. ChrisA Sorry for any injected confusion here, but that line "data = sorted(data)" appears as though it takes the value of the variable named _data_, sorts it and returns it to the same variable store, so no copy would be created. Am I missing something there? -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On Wed, 29 Mar 2023 at 16:56, Greg Ewing via Python-list wrote: > > On 28/03/23 2:25 pm, Travis Griggs wrote: > > Interestingly the error also only started showing up when I switched from > > running a statistics.mean() on one of these, instead of what I had been > > using, a statistics.median(). Apparently the kind of iteration done in a > > mean, is more conflict prone than a median? > > It may be a matter of whether the GIL is held or not. I had a look > at the source for deque, and it doesn't seem to explicitly do > anything about locking, it just relies on the GIL. > > So maybe statistics.median() is implemented in C and statistics.mean() > in Python, or something like that? > Both functions are implemented in Python, but median() starts out with this notable line: data = sorted(data) which gives back a copy, iterated over rapidly in C. All subsequent work is done on that copy. The same effect could be had with mean() by taking a snapshot using list(q) and, I believe, would have the same effect (the source code for the sorted() function begins by calling PySequence_List). In any case, it makes *conceptual* sense to do your analysis on a copy of the queue, thus ensuring that your stats are stable. The other threads can keep going while you do your calculations, even if that means changing the queue. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On 28/03/23 2:25 pm, Travis Griggs wrote: Interestingly the error also only started showing up when I switched from running a statistics.mean() on one of these, instead of what I had been using, a statistics.median(). Apparently the kind of iteration done in a mean, is more conflict prone than a median? It may be a matter of whether the GIL is held or not. I had a look at the source for deque, and it doesn't seem to explicitly do anything about locking, it just relies on the GIL. So maybe statistics.median() is implemented in C and statistics.mean() in Python, or something like that? -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On 2023-03-27 at 18:25:01 -0700, Travis Griggs wrote: > "Deques support thread-safe, memory efficient appends and pops from > either side of the deque with approximately the same O(1) performance > in either direction.” > (https://docs.python.org/3.11/library/collections.html?highlight=deque#collections.deque) [...] > I guess this surprised me. When I see “thread safe”, I don’t expect to > get errors. Even without threads, mutating a collection while iterating over it usually results in bad things happening. $ python Python 3.10.10 (main, Mar 5 2023, 22:26:53) [GCC 12.2.1 20230201] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import collections >>> x = collections.deque() >>> x.append(44) >>> x.append(55) >>> x.append(66) >>> x.append(77) >>> x deque([44, 55, 66, 77]) >>> for y in x: x.pop() 77 Traceback (most recent call last): File "", line 1, in RuntimeError: deque mutated during iteration Concurrency just increases the likeliness of mutating while iterating. -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On 2023-03-28, Travis Griggs wrote: > A while ago I chose to use a deque that is shared between two threads. I did > so because the docs say: > > "Deques support thread-safe, memory efficient appends and pops from > either side of the deque with approximately the same O(1) > performance in either direction.” > > (https://docs.python.org/3.11/library/collections.html?highlight=deque#collections.deque) > > Earlier today, looking through some server logs I noticed that from > time to I’m getting a > > RuntimeError: deque mutated during iteration > > I guess this surprised me. When I see “thread safe”, I don’t expect > to get errors. Well, I guess it doesn't say that iteration of a deque is thread safe. It only claims that appends and pops from either end are thread safe. It doesn't even claim that inserts, removes, clear, copy, or any other operations are thread-safe. -- Grant -- https://mail.python.org/mailman/listinfo/python-list
Re: What kind of "thread safe" are deque's actually?
On Tue, 28 Mar 2023 at 12:26, Travis Griggs wrote: > > A while ago I chose to use a deque that is shared between two threads. I did > so because the docs say: > > "Deques support thread-safe, memory efficient appends and pops from either > side of the deque with approximately the same O(1) performance in either > direction.” > > (https://docs.python.org/3.11/library/collections.html?highlight=deque#collections.deque) > > Earlier today, looking through some server logs I noticed that from time to > I’m getting a > > RuntimeError: deque mutated during iteration > > I guess this surprised me. When I see “thread safe”, I don’t expect to get > errors. > I'd like to see your code, but here's an example with no threads whatsoever that has the same error: >>> from collections import deque >>> q = deque([1, 2, 3, 4]) >>> for item in q: ... if item % 2: q.append(item * 2) ... print(item) ... 1 Traceback (most recent call last): File "", line 1, in RuntimeError: deque mutated during iteration This error comes from having an active iterator, then mutating the deque, then continuing to iterate. That MAY be a result of threading, but it isn't necessarily. For threaded usage, I would recommend restricting yourself to append/appendleft/pop/popleft (the standard mutators), with any operations on the entire queue being done on a copy instead (either q.copy() or list(q) depending on what you need). The act of taking a copy should itself be thread-safe, and obviously anything done on a separate copy will be independent of changes to the original. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
What kind of "thread safe" are deque's actually?
A while ago I chose to use a deque that is shared between two threads. I did so because the docs say: "Deques support thread-safe, memory efficient appends and pops from either side of the deque with approximately the same O(1) performance in either direction.” (https://docs.python.org/3.11/library/collections.html?highlight=deque#collections.deque) Earlier today, looking through some server logs I noticed that from time to I’m getting a RuntimeError: deque mutated during iteration I guess this surprised me. When I see “thread safe”, I don’t expect to get errors. Interestingly the error also only started showing up when I switched from running a statistics.mean() on one of these, instead of what I had been using, a statistics.median(). Apparently the kind of iteration done in a mean, is more conflict prone than a median? I’ve got a couple ways I can work around this. But I was surprised. -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
“So the case where the assumption fails may not be easily reproducable and the more information you can get post-mortem the better” That’s true for rare corner cases or esoteric race conditions. Usually, when I see asserts it's just because I was just plain stupid. From: Python-list on behalf of Peter J. Holzer Date: Saturday, February 25, 2023 at 5:21 PM To: python-list@python.org Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ? On 2023-02-25 21:58:18 +, Weatherby,Gerard wrote: > I only use asserts for things I know to be true. Yeah, that's what assers are for. Or rather for things that you *think* are true. > In other words, a failing assert means I have a hole in my program > logic. Yes, if you include your assumptions in your definition of "logic". > For that use, the default behavior –telling me which line the assert > is on, is more than sufficient. Depending on the circumstance, I’ll > re-run the code with a breakpoint or replace the assert with an > informative f-string Exception. That may not always be practical. Things that we know (or think) are true often have *are* true in most cases (otherwise we wouldn't think so). So the case where the assumption fails may not be easily reproducable and the more information you can get post-mortem the better. For example, in C on Linux a failed assertion causes a core dump. So you can inspect the complete state of the program. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-25 21:58:18 +, Weatherby,Gerard wrote: > I only use asserts for things I know to be true. Yeah, that's what assers are for. Or rather for things that you *think* are true. > In other words, a failing assert means I have a hole in my program > logic. Yes, if you include your assumptions in your definition of "logic". > For that use, the default behavior –telling me which line the assert > is on, is more than sufficient. Depending on the circumstance, I’ll > re-run the code with a breakpoint or replace the assert with an > informative f-string Exception. That may not always be practical. Things that we know (or think) are true often have *are* true in most cases (otherwise we wouldn't think so). So the case where the assumption fails may not be easily reproducable and the more information you can get post-mortem the better. For example, in C on Linux a failed assertion causes a core dump. So you can inspect the complete state of the program. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
I only use asserts for things I know to be true. Nothing is harder to debug
than when something you know to be true turns out to be… not True. Because I’ll
check everything else instead of the cause of the bug.
In other words, a failing assert means I have a hole in my program logic.
For that use, the default behavior –telling me which line the assert is on, is
more than sufficient. Depending on the circumstance, I’ll re-run the code with
a breakpoint or replace the assert with an informative f-string Exception.
From: Python-list on
behalf of Peter J. Holzer
Date: Saturday, February 25, 2023 at 9:22 AM
To: python-list@python.org
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg)
values are ?
On 2023-02-25 09:10:06 -0500, Thomas Passin wrote:
> On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> > On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> > > Sometimes you can use a second parameter to assert if you know what kind
> > > of
> > > error to expect:
[...]
> > > With type errors, assert may actually give you the information needed:
> > >
> > > > > > c = {"a": a, "b": 2}
> > > > > > assert a > c
> > > Traceback (most recent call last):
> > >File "", line 1, in
> > > TypeError: '>' not supported between instances of 'list' and 'dict'
> >
> > Actually in this case it isn't assert which gives you the information,
> > it's evaluating the expression itself. You get the same error with just
> > a > c
> > on a line by its own.
>
> In some cases. For my example with an explanatory string, you wouldn't want
> to write code like that after an ordinary line of code, at least not very
> often. The assert statement allows it syntactically.
Yes, but if an error in the expression triggers an exception (as in this
case) the explanatory string will never be displayed.
hp
--
_ | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| | | h...@hjp.at |-- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-25 09:10:06 -0500, Thomas Passin wrote:
> On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> > On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> > > Sometimes you can use a second parameter to assert if you know what kind
> > > of
> > > error to expect:
[...]
> > > With type errors, assert may actually give you the information needed:
> > >
> > > > > > c = {"a": a, "b": 2}
> > > > > > assert a > c
> > > Traceback (most recent call last):
> > >File "", line 1, in
> > > TypeError: '>' not supported between instances of 'list' and 'dict'
> >
> > Actually in this case it isn't assert which gives you the information,
> > it's evaluating the expression itself. You get the same error with just
> > a > c
> > on a line by its own.
>
> In some cases. For my example with an explanatory string, you wouldn't want
> to write code like that after an ordinary line of code, at least not very
> often. The assert statement allows it syntactically.
Yes, but if an error in the expression triggers an exception (as in this
case) the explanatory string will never be displayed.
hp
--
_ | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| | | h...@hjp.at |-- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
signature.asc
Description: PGP signature
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
On 2/24/2023 2:47 PM, dn via Python-list wrote:
On 25/02/2023 08.12, Peter J. Holzer wrote:
On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
In some ways, providing this information seems appropriate.
Curiously, this does not even occur during an assert exception -
despite the value/relationship being the whole point of using
the command!
x = 1
assert x == 2
AssertionError (and that's it)
Sometimes you can use a second parameter to assert if you know what kind of
error to expect:
a = [1,2,3]
b = [4,5]
assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
Traceback (most recent call last):
File "", line 1, in
AssertionError: len(a): 3 != len(b): 2
Yup. That's very useful (but I tend to forget that).
With type errors, assert may actually give you the information needed:
c = {"a": a, "b": 2}
assert a > c
Traceback (most recent call last):
File "", line 1, in
TypeError: '>' not supported between instances of 'list' and 'dict'
Actually in this case it isn't assert which gives you the information,
it's evaluating the expression itself. You get the same error with just
a > c
on a line by its own.
In some cases. For my example with an explanatory string, you wouldn't
want to write code like that after an ordinary line of code, at least
not very often. The assert statement allows it syntactically.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> On 2/24/2023 2:47 PM, dn via Python-list wrote:
> > On 25/02/2023 08.12, Peter J. Holzer wrote:
> > > On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
> > > > In some ways, providing this information seems appropriate.
> > > > Curiously, this does not even occur during an assert exception -
> > > > despite the value/relationship being the whole point of using
> > > > the command!
> > > >
> > > > x = 1
> > > > assert x == 2
> > > >
> > > > AssertionError (and that's it)
>
> Sometimes you can use a second parameter to assert if you know what kind of
> error to expect:
>
> >>> a = [1,2,3]
> >>> b = [4,5]
> >>> assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
> Traceback (most recent call last):
> File "", line 1, in
> AssertionError: len(a): 3 != len(b): 2
Yup. That's very useful (but I tend to forget that).
> With type errors, assert may actually give you the information needed:
>
> >>> c = {"a": a, "b": 2}
> >>> assert a > c
> Traceback (most recent call last):
> File "", line 1, in
> TypeError: '>' not supported between instances of 'list' and 'dict'
Actually in this case it isn't assert which gives you the information,
it's evaluating the expression itself. You get the same error with just
a > c
on a line by its own.
hp
--
_ | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| | | h...@hjp.at |-- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
signature.asc
Description: PGP signature
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2/24/2023 2:47 PM, dn via Python-list wrote:
On 25/02/2023 08.12, Peter J. Holzer wrote:
On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
In some ways, providing this information seems appropriate.
Curiously, this
does not even occur during an assert exception - despite the
value/relationship being the whole point of using the command!
x = 1
assert x == 2
AssertionError (and that's it)
Sometimes you can use a second parameter to assert if you know what kind
of error to expect:
>>> a = [1,2,3]
>>> b = [4,5]
>>> assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
Traceback (most recent call last):
File "", line 1, in
AssertionError: len(a): 3 != len(b): 2
With type errors, assert may actually give you the information needed:
>>> c = {"a": a, "b": 2}
>>> assert a > c
Traceback (most recent call last):
File "", line 1, in
TypeError: '>' not supported between instances of 'list' and 'dict'
So now we know that a is a list and c is a dictionary.
Pytest is great there. If an assertion in a test case fails it analyzes
the expression to give you various levels of details:
test session starts
platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /home/hjp/tmp/t
plugins: cov-3.0.0, anyio-3.6.1
collected 1 item
test_a.py
F [100%]
= FAILURES
==
__ test_a
___
def test_a():
a = [1, 2, 3]
b = {"a": a, "b": 2}
assert len(a) == len(b)
E AssertionError: assert 3 == 2
E + where 3 = len([1, 2, 3])
E + and 2 = len({'a': [1, 2, 3], 'b': 2})
test_a.py:7: AssertionError
== short test summary info
==
FAILED test_a.py::test_a - AssertionError: assert 3 == 2
= 1 failed in 0.09s
=
+1
and hence the tone of slight surprise in the observation - because only
ever use assert within pytests, and as observed, pytest amplifies the
report-back to provide actionable-intelligence. See also: earlier
contribution about using a debugger.
That said, have observed coders 'graduating' from other languages,
making wider use of assert - assumed to be more data (value)
sanity-checks than typing, but ...
Do you use assert frequently?
The OP seems wedded to his?her ways, complaining that Python does not
work the way it 'should'. In turn, gives rise to the impression that
expounding the advantages of TDD, and thus anticipating such unit and
integration error-possibilities, might be considered an insult or
unhelpful.
(sigh!)
Personally, I struggled a bit to adapt from the more-strictured (if not
more-structured) languages of my past, to Python - particularly the
different philosophies or emphases of what happens at 'compile-time' cf
'execution-time'; and how such required marked changes in attitudes to
design, time-allocation, work-flow, and tool-set. Two related-activities
which made the language-change more workable and unleashed greater than
step-change advantage, were: increased use of TDD, and actively learning
the facilities within Python-oriented IDEs.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-25 08:47:00 +1300, dn via Python-list wrote: > That said, have observed coders 'graduating' from other languages, making > wider use of assert - assumed to be more data (value) sanity-checks than > typing, but ... > > Do you use assert frequently? Not very often, but I do use it. Sometimes for its intended purpose (i.e. to guard against bugs or wrong assumptions), sometimes just to guard incomplete or sloppy code (e.g. browsing through some projects I find assert len(data["structure"]["dimensions"]["observation"]) == 1 (incomplete code - I didn't bother to implement multiple observations) and assert(header[0] == "Monat (MM)") (the code below is sloppy. Instead of fixing it I just made the original programmer's assumptions explicit) and of course assert False (this point should never be reached)). hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 25/02/2023 08.12, Peter J. Holzer wrote:
On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
In some ways, providing this information seems appropriate. Curiously, this
does not even occur during an assert exception - despite the
value/relationship being the whole point of using the command!
x = 1
assert x == 2
AssertionError (and that's it)
Pytest is great there. If an assertion in a test case fails it analyzes
the expression to give you various levels of details:
test session starts
platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /home/hjp/tmp/t
plugins: cov-3.0.0, anyio-3.6.1
collected 1 item
test_a.py F
[100%]
= FAILURES
==
__ test_a
___
def test_a():
a = [1, 2, 3]
b = {"a": a, "b": 2}
assert len(a) == len(b)
E AssertionError: assert 3 == 2
E+ where 3 = len([1, 2, 3])
E+ and 2 = len({'a': [1, 2, 3], 'b': 2})
test_a.py:7: AssertionError
== short test summary info
==
FAILED test_a.py::test_a - AssertionError: assert 3 == 2
= 1 failed in 0.09s
=
+1
and hence the tone of slight surprise in the observation - because only
ever use assert within pytests, and as observed, pytest amplifies the
report-back to provide actionable-intelligence. See also: earlier
contribution about using a debugger.
That said, have observed coders 'graduating' from other languages,
making wider use of assert - assumed to be more data (value)
sanity-checks than typing, but ...
Do you use assert frequently?
The OP seems wedded to his?her ways, complaining that Python does not
work the way it 'should'. In turn, gives rise to the impression that
expounding the advantages of TDD, and thus anticipating such unit and
integration error-possibilities, might be considered an insult or unhelpful.
(sigh!)
Personally, I struggled a bit to adapt from the more-strictured (if not
more-structured) languages of my past, to Python - particularly the
different philosophies or emphases of what happens at 'compile-time' cf
'execution-time'; and how such required marked changes in attitudes to
design, time-allocation, work-flow, and tool-set. Two related-activities
which made the language-change more workable and unleashed greater than
step-change advantage, were: increased use of TDD, and actively learning
the facilities within Python-oriented IDEs.
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
> In some ways, providing this information seems appropriate. Curiously, this
> does not even occur during an assert exception - despite the
> value/relationship being the whole point of using the command!
>
> x = 1
> assert x == 2
>
> AssertionError (and that's it)
Pytest is great there. If an assertion in a test case fails it analyzes
the expression to give you various levels of details:
test session starts
platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /home/hjp/tmp/t
plugins: cov-3.0.0, anyio-3.6.1
collected 1 item
test_a.py F
[100%]
= FAILURES
==
__ test_a
___
def test_a():
a = [1, 2, 3]
b = {"a": a, "b": 2}
> assert len(a) == len(b)
E AssertionError: assert 3 == 2
E+ where 3 = len([1, 2, 3])
E+ and 2 = len({'a': [1, 2, 3], 'b': 2})
test_a.py:7: AssertionError
== short test summary info
==
FAILED test_a.py::test_a - AssertionError: assert 3 == 2
= 1 failed in 0.09s
=
hp
--
_ | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| | | h...@hjp.at |-- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
signature.asc
Description: PGP signature
--
https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-23 20:32:26 -0700, Michael Torrie wrote: > On 2/23/23 01:08, Hen Hanna wrote: > > Python VM is seeing an "int" object (123) (and telling me that) > > ... so it should be easy to print that "int" object What does > > Python VMknow ? and when does it know it ? > It knows there is an object and its name and type. It knows this from > the first moment you create the object and bind a name to it. > > it seems like it is being playful, teasing (or mean),and > > hiding the ball from me > > Sorry you aren't understanding. Whenever you print() out an object, > python calls the object's __repr__() method to generate the string to > display. For built-in objects this is obviously trivial. But if you > were dealing an object of some arbitrary class, there may not be a > __repr__() method Is this even possible? object has a __repr__ method, so all other classes would inherit that if they don't define one themselves. I guess it's possible to explicitely remove it ... > which would cause an exception, or if the __repr__() > method itself raised an exception, Yup. That is possible and has happened to me several times - of course always in a situation where I really needed that output ... hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-22 15:46:09 -0800, Hen Hanna wrote: > On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: > > > py bug.py > > Traceback (most recent call last): > > File "C:\Usenet\bug.py", line 5, in > > print( a + 12 ) > > TypeError: can only concatenate str (not "int") to str > > > > > > Why doesn't Python (error msg) do the obvious thing and tell me > > WHAT the actual (offending, arg) values are ? > > > > In many cases, it'd help to know what string the var A had , when the error > > occurred. > > i wouldn't have to put print(a) just above, to see. > > > > ( pypy doesn't do that either, but Python makes programming > > (debugging) so easy that i hardly feel any inconvenience.) That seems like a non-sequitur to me. If you hardly feel any inconvenience, why argue so forcefully? And why is pypy relevant here? > i see that my example would be clearER with this one-line change: > > > > py bug.py > >Traceback (most recent call last): > > File "C:\Usenet\bug.py", line 5, in > map( Func,fooBar( X, Y, X + > Y )) > >TypeError: can only concatenate str (not "int") to str > > > i hope that NOW a few of you can see this as a genuine, (reasonable) > question. That doesn't seem a better example to me. There is still only one subexpression (X + Y) where that error can come from, so I know that X is a str and Y is an int. A better example would be something like x = (a + b) * (c + d) In this case it could be either (a + b) or (c + d) which caused the error. But what I really want to know here is the names of the involved variables, NOT the values. If the error message told me that the values were 'foo' and 12.3, I still wouldn't be any wiser. The problem here of course is that the operands aren't necessarily simple variables as in this example - they may be arbitrarily complex expressions. However, it might be sufficient to mark the operator which caused the exception: | ... | File "/home/hjp/tmp/./foo", line 4, in f | return (a + b) * (c + d) | ^ | TypeError: can only concatenate str (not "int") to str would tell me that (c + d) caused the problem and therefore that c must be a str which it obviously shouldn't be. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
RE: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
We have been supplying many possible reasons or consequences for why the implementation of python does not do what the OP wants and even DEMANDS. I am satisfied with knowing it was because they CHOSE NOT TO in some places and maybe not in others. It is nice to see some possible reasons, but something as simple as efficiency or needing to complicate the code in something used regularly, might be enough for now. But to comment on what Michael T. and Dave N. have been saying, newcomers often have no clue of what can happen so their questions may sound quite reasonable. So what happens if you create a large data structure, so some operation that fails, catch the error and save the variables involved in an exception and throw that onward and perhaps the program keeps running? There is now a pointer to the large data structure in the exception object, or even a copy. If that exception is not discarded or garbage collected, it can remain in memory indefinitely even if the original string was expected to be removed, replaced, or garbage collected. Some modern features in R such as generators will stay alive infinitely and retain their state in between calls for a next item. You can end up with memory leaks that are not trivial to solve or that may mysteriously disappear when an iterable has finally been consumed and all the storage it used or pointed at can be retrieved, as one example. A more rational approach is to realize that python has multiple levels of debugging and exceptions are one among many. They are not meant to solve the entire problem but just enough to be helpful or point you in some direction. Yes, they can do more. And, FYI, I too pointed this person at the Tutor list and I see no sign they care how many people they make waste their time with so many mainly gripes. I personally now ignore any post by them. -Original Message- From: Python-list On Behalf Of Michael Torrie Sent: Thursday, February 23, 2023 10:32 PM To: python-list@python.org Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ? On 2/23/23 01:08, Hen Hanna wrote: > Python VM is seeing an "int" object (123) (and telling me that) ... so it should be easy to print that "int" object > What does Python VMknow ? and when does it know it ? It knows there is an object and its name and type. It knows this from the first moment you create the object and bind a name to it. > it seems like it is being playful, teasing (or mean),and hiding the ball from me Sorry you aren't understanding. Whenever you print() out an object, python calls the object's __repr__() method to generate the string to display. For built-in objects this is obviously trivial. But if you were dealing an object of some arbitrary class, there may not be a __repr__() method which would cause an exception, or if the __repr__() method itself raised an exception, you'd lose the original error message and the stack trace would be all messed up and of no value to you. Does that make sense? Remember that Python is a very dynamic language and what might be common sense for a built-in type makes no sense at all for a custom type. Thus there's no consistent way for Python to print out the information you think is so simple. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2/23/23 01:08, Hen Hanna wrote: > Python VM is seeing an "int" object (123) (and telling me that) ... > so it should be easy to print that "int" object > What does Python VMknow ? and when does it know it ? It knows there is an object and its name and type. It knows this from the first moment you create the object and bind a name to it. > it seems like it is being playful, teasing (or mean),and hiding the > ball from me Sorry you aren't understanding. Whenever you print() out an object, python calls the object's __repr__() method to generate the string to display. For built-in objects this is obviously trivial. But if you were dealing an object of some arbitrary class, there may not be a __repr__() method which would cause an exception, or if the __repr__() method itself raised an exception, you'd lose the original error message and the stack trace would be all messed up and of no value to you. Does that make sense? Remember that Python is a very dynamic language and what might be common sense for a built-in type makes no sense at all for a custom type. Thus there's no consistent way for Python to print out the information you think is so simple. -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 23/02/2023 09.05, Hen Hanna wrote:
> py bug.py
Traceback (most recent call last):
File "C:\Usenet\bug.py", line 5, in
print( a + 12 )
TypeError: can only concatenate str (not "int") to str
Why doesn't Python (error msg) do the obvious thing and tell me
WHAT the actual (offending, arg) values are ?
In many cases, it'd help to know what string the var A had , when the
error occurred.
i wouldn't have to put print(a) just
above, to see.
In some ways, providing this information seems appropriate. Curiously,
this does not even occur during an assert exception - despite the
value/relationship being the whole point of using the command!
x = 1
assert x == 2
AssertionError (and that's it)
Then again, remember that exceptions can be 'caught'. So, such data
would need to be added to the exception-instance. This could become
quite costly.
What are the appropriate tools for the job?
Don't add an extra print(), use a debugger.
Not only does this allow you to breakpoint critical points in the code,
but identifiers can be watch-ed and changes noted. The other handy
feature is being able to correct the current erroneous value of the
identifier and continue execution.
For us, memory-challenged coders, there is no need to remember to remove
the print() again, afterwards.
The TypeError indicates a problem between the programmer's ears. What
was thought to be a string or an integer was the opposite. This seems to
be playing fast-and-loose with Python's dynamic-typing. To quote: "we're
all adults here". Thus, I wouldn't recommend 're-cycling' an identifier
to represent two different (types of) data-point in the same code - but
there's nothing to stop you/anyone!
The other possibility is that it was an accident. Sounds more like
something I would do, but... In this case, the tool which is your/my
friend is typing. The IDE should catch most of the situations where an
int would be used as an str, or v-v. Remember though, Python's typing is
(a) not part of the language, and (b) probably won't help at run-time.
PS are you aware that there is a Python-Tutor list for the use of people
learning Python?
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list
RE: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
Rob, There are lots of nifty features each of us might like and insist make much more sense than what others say they want. Sometimes the answer is to not satisfy most of those demands but provide TOOLS they can use to do things for themselves. As you agree, many of us have found all kinds of tools that help with debugging and frankly, some of them use it to the point of annoying others who would rather avoid them. An example is type hints that can get quite detailed and obscure the outline of your program and are ignored by the parser and only relevant for some linter or other such program. I am thinking of what would happen if I created several fairly long or complex data structures and tried to add them. Say I have a dictionary containing millions of entries including every conceivable UNICODE character as well as very complex values such as other dictionaries or strings containing entire books. My other data structure might be a forest containing many smaller trees, such as the output of some machine learning models or perhaps showing every possible game of chess up to 50 moves deep along with an evaluation of the relative strength of the board to one player. I then accidentally write code that contains: big_dic + forest_trees Would I like my error message consume all the paper in my city (or scroll my screen for a week) as it tells me a dict cannot be added to a forest and by the way, here is a repr of each of them showing the current (highly recursive) contents. Now people have written functions that take something long and truncate it so a list containing [1, 2, 3, ... 1_000_000] is shown in this condensed form with the rest missing, but then someone will complain they are not seeing all of it! So the deal is to use your TOOLS. You can run a debugger or add print statements or enclose it in a try/catch to keep it from stopping the program and other techniques. You can examine the objects carefully just before, or even after and do cautious things like ask for the length and then maybe ask for the first few and last few items, or whatever makes sense. In the original example, we were first asked about print(a + b) and later given a somewhat weirder func(x, y, x +y) as examples. Now ask what order things are evaluated and where the error happens. What is known by the party handling the error? If you put the offending statement in a try/catch scenario, then when the error is triggered, YOU wrote the code that catches the exception and you can often examine the payload of the exception, or know that the arguments of a or b or x or y were involved and you can craft your own output to be more clear. Or, you can even sometimes fix the problem and redo the code using something like float(x) or str(y). My impression here is that the error is not on the surface but caught deeper. The symbols used may be x and y but what if we work with "12" + 13 and follow what happens? Since the interpreter in python evaluates x+y before calling the function using the result as an argument, the function never sees anything. What should happen is that the interpreter sees a "12" which is normally an object of a class of str and then it sees a "+" and then it sees anything that follows as a second object it ignores for now. Python does not have a fully defined operator that it invokes when it sees a "+" as the meaning depends on what object is being asked to do whatever plus means to it. For a string argument, it means concatenate to your current content and return a new str object. The way that happens is that the class (or a relative) has defined a method called __add__() or it hasn't. If it has, it takes an argument of the second object and in this case it gets the integer object containing 13. So it runs the function and it has not been programmed on how to append an integer to a string of characters and it returns without an answer but an exception. The interpreter evaluator does not admit defeat yet and reasonable tries to see if the integer 13 has a __iadd__() which is similar but different and again, an integer has not been programmed to append itself to an object of type str. Could it have been? Sure. If you make your own (sub)class you can create a kind of integer that will make a str version of itself and append it o the "12" to make "1213" BUT in this case, that is not an option. So the integer method fails and returns an exception too. Now the parser functionality knows it has failed. "12" and 13 have both refused to implement the plus sign and either it catches the exception OR is does not and lets it flow upstream till any other functions in a chain catch it. Any one can then generate some error message, or it can reach the top level of the interpreter and it has to decide what to do. But some errors are not fatal. If str had no __add__() that is not an error. If it returns that it cannot do it, that is not a fatal err
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 22/02/2023 20:05, Hen Hanna wrote: Python makes programming (debugging) so easy I agree with that! Rob Cliffe -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On Wednesday, February 22, 2023 at 11:57:45 PM UTC-8, Barry wrote: > > On 23 Feb 2023, at 01:39, Hen Hanna wrote: > > > > On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote: > >> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: > >>>> py bug.py > >>> Traceback (most recent call last): > >>> File "C:\Usenet\bug.py", line 5, in > >>> print( a + 12 ) > >>> TypeError: can only concatenate str (not "int") to str > >>> > >>> > >>> Why doesn't Python (error msg) do the obvious thing and tell me > >>> WHAT the actual (offending, arg) values are ? > >>> > >>> In many cases, it'd help to know what string the var A had , when the > >>> error occurred. > >>> i wouldn't have to put print(a) just above, to see. > >>> > >>> > >>> > >>> > >>> ( pypy doesn't do that either, but Python makes programming (debugging) > >>> so easy that i hardly feel any inconvenience.) > > > > > > i see that my example would be (even) clearER with this one-line change: > > > > py bug.py > > > > Traceback (most recent call last): > > > > File "C:\Usenet\bug.py", line 5, in > > map( Func, fooBar( X, Y, X + Y )) > > > > TypeError: can only concatenate str (not "int") to str > > attempt to call + with 'abc', 123 > > <-- > > > >> i hope that NOW a few of you can see this as a genuine, (reasonable) > >> question. > > > > Python seems so perfectly User-friendly that > > i 'm so curious (puzzled) that it doesn't do the very obvious and easy > > thing > > of giving me this info: > > > > attempt to call +with 'abc', > > 123 <-- > It is not easy to do that in a robust and reliable way for any object. > You can end up in the code to generate the error message itself breaking. > For example using unbounded CPU time when attempting to get the string repr > of the variable. > > Barry > Python VM is seeing an "int" object (123) (and telling me that) ... so it should be easy to print that "int" object What does Python VMknow ? and when does it know it ? it seems like it is being playful, teasing (or mean),and hiding the ball from me -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
> On 23 Feb 2023, at 01:39, Hen Hanna wrote: > > On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote: >> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: >>>> py bug.py >>> Traceback (most recent call last): >>> File "C:\Usenet\bug.py", line 5, in >>> print( a + 12 ) >>> TypeError: can only concatenate str (not "int") to str >>> >>> >>> Why doesn't Python (error msg) do the obvious thing and tell me >>> WHAT the actual (offending, arg) values are ? >>> >>> In many cases, it'd help to know what string the var A had , when the error >>> occurred. >>> i wouldn't have to put print(a) just above, to see. >>> >>> >>> >>> >>> ( pypy doesn't do that either, but Python makes programming (debugging) so >>> easy that i hardly feel any inconvenience.) > > > i see that my example would be (even) clearER with this one-line change: > > py bug.py > > Traceback (most recent call last): > > File "C:\Usenet\bug.py", line 5, in > map( Func, fooBar( X, Y, X + Y )) > > TypeError: can only concatenate str (not "int") to str >attempt to call + with 'abc' , 123.45 > <-- > >> i hope that NOW a few of you can see this as a genuine, (reasonable) >> question. > > Python seems so perfectly User-friendly that > i 'm so curious (puzzled) that it doesn't do the very > obvious and easy thing > of giving me this info: > >attempt to call + with 'abc' , > 123.45 <-- It is not easy to do that in a robust and reliable way for any object. You can end up in the code to generate the error message itself breaking. For example using unbounded CPU time when attempting to get the string repr of the variable. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote: > On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: > > > py bug.py > > Traceback (most recent call last): > > File "C:\Usenet\bug.py", line 5, in > > print( a + 12 ) > > TypeError: can only concatenate str (not "int") to str > > > > > > Why doesn't Python (error msg) do the obvious thing and tell me > > WHAT the actual (offending, arg) values are ? > > > > In many cases, it'd help to know what string the var A had , when the error > > occurred. > > i wouldn't have to put print(a) just above, to see. > > > > > > > > > > ( pypy doesn't do that either, but Python makes programming (debugging) so > > easy that i hardly feel any inconvenience.) i see that my example would be (even) clearER with this one-line change: py bug.py Traceback (most recent call last): File "C:\Usenet\bug.py", line 5, in map( Func, fooBar( X, Y, X + Y )) TypeError: can only concatenate str (not "int") to str attempt to call + with 'abc' , 123.45 <-- > i hope that NOW a few of you can see this as a genuine, (reasonable) question. Python seems so perfectly User-friendly that i 'm so curious (puzzled) that it doesn't do the very obvious and easy thing of giving me this info: attempt to call + with 'abc' , 123.45 <-- -- https://mail.python.org/mailman/listinfo/python-list
