RE: Terminal Emulator
The topic was to re-invent the wheel yet again and create a terminal emulator. I hesitate to say this but one approach is to consider the curses module as described by our very own Alan Gauld in a book: https://www.amazon.com/Programming-curses-Python-Alan-Gauld-ebook/dp/B091B85 B77 The topic is how to make a terminal emulator and as Alan mentions, each kind of terminal may accept various kinds of escape sequences. There are files available the are normally used by curses to get a description of sorts of the capabilities and details of a terminal like a VT100 that curses can use to decide what stream of bytes to send to update a screen. You might be able to use something similar, or better, to see what your terminal emulator should emulate. And, it may even be possible for you to emulate lots of terminals with the same basic code. -Original Message- From: Python-list On Behalf Of Alan Gauld via Python-list Sent: Tuesday, May 14, 2024 3:07 PM To: Gordinator ; python-list@python.org Subject: Re: Terminal Emulator On 14/05/2024 18:44, Gordinator via Python-list wrote: > I wish to write a terminal emulator in Python. I am a fairly competent > Python user, and I wish to try a new project idea. What references can I > use when writing my terminal emulator? I wish for it to be a true > terminal emulator as well, not just a Tk text widget or something like that. The first thing is to decide which terminal. A VT100 is very different from a 3270. And even a VT330 is quite different from a VT100 although sharing a common subset of control codes. And if you start looking at graphical terminals things get even more interesting! The other thing to consider is whether it will be a standalone app or a GUI component. If the latter do you want to expose your own API or clone the manufacturers? Or both?! Or you could make it an object that can be used both in GUIs and in "robotic" or "batch" mode. So many options. Most of the specs are available online and there must be dozens of terminal emulators around written in C so you should have plenty of sample code to study. Good luck! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: A technique from a chatbot
That is an excellent point, Mark. Some of the proposed variants to the requested problem, including mine, do indeed find all instances only to return the first. This can use additional time and space but when done, some of the overhead is also gone. What I mean is that a generator you create and invoke once, generally sits around indefinitely in your session unless it leaves your current range or something. It does only a part of the work and must remain suspended and ready to be called again to do more. If you create a generator inside a function and the function returns, presumably it can be garbage-collected. But if it is in the main body, I have to wonder what happen. There seem to be several related scenarios to consider. - You may want to find, in our example, a first instance. Right afterwards, you want the generator to disassemble anything in use. - You may want the generator to stick around and later be able to return the next instance. The generator can only really go away when another call has been made after the last available instance and it cannot look for more beyond some end. - Finally, you can call a generator with the goal of getting all instances such as by asking it to populate a list. In such a case, you may not necessarily want or need to use a generator expression and can use something straightforward and possible cheaper. What confuses the issue, for me, is that you can make fairly complex calculations in python using various forms of generators that implement a sort of just-in-time approach as generators call other generators which call yet others and so on. Imagine having folders full of files that each contain a data structure such as a dictionary or set and writing functionality that searches for the first match for a key in any of the dictionaries (or sets or whatever) along the way? Now imagine that dictionary items can be a key value pair that can include the value being a deeper dictionary, perhaps down multiple levels. You could get one generator that generates folder names or opens them and another that generates file names and reads in the data structure such as a dictionary and yet another that searches each dictionary and also any internally embedded dictionaries by calling another instance of the same generator as much as needed. You can see how this creates and often consumes generators along the way as needed and in a sense does the minimum amount of work needed to find a first instance. But what might it leave open and taking up resources if not finished in a way that dismantles it? Perhaps worse, imagine doing the search in parallel and as sone as it is found anywhere, ... -Original Message- From: Python-list On Behalf Of Mark Bourne via Python-list Sent: Thursday, April 4, 2024 3:04 PM To: python-list@python.org Subject: Re: A technique from a chatbot Thomas Passin wrote: > On 4/2/2024 1:47 PM, Piergiorgio Sartor via Python-list wrote: >> On 02/04/2024 19.18, Stefan Ram wrote: >>>Some people can't believe it when I say that chatbots improve >>>my programming productivity. So, here's a technique I learned >>>from a chatbot! >>>It is a structured "break". "Break" still is a kind of jump, >>>you know? >>>So, what's a function to return the first word beginning with >>>an "e" in a given list, like for example >>> [ 'delta', 'epsilon', 'zeta', 'eta', 'theta' ] >>> >>>? Well it's >>> def first_word_beginning_with_e( list_ ): >>> for word in list_: >>> if word[ 0 ]== 'e': return word >>> >>>. "return" still can be considered a kind of "goto" statement. >>>It can lead to errors: >>> >>> def first_word_beginning_with_e( list_ ): >>> for word in list_: >>> if word[ 0 ]== 'e': return word >>> something_to_be_done_at_the_end_of_this_function() >>>The call sometimes will not be executed here! >>>So, "return" is similar to "break" in that regard. >>>But in Python we can write: >>> def first_word_beginning_with_e( list_ ): >>> return next( ( word for word in list_ if word[ 0 ]== 'e' ), None ) >> >> Doesn't look a smart advice. >> >>>. No jumps anymore, yet the loop is aborted on the first hit > > It's worse than "not a smart advice". This code constructs an > unnecessary tuple, then picks out its first element and returns that. I don't think there's a tuple being created. If you mean: ( word for word in list_ if word[ 0 ]== 'e' ) ...that's not creating a tuple. It's a generator expression, which generates the next value each time it's called for. If you only ever ask for the first item, it only generates that one. When I first came across them, I did find it a bit odd that generator expressions look like the tuple equivalent of list/dictionary comprehensions. FWIW, if you actually wanted a tuple from that expression, you'd need to pass the generator to tuple's constructor: tuple(word for word in list_ if word[0]
RE: A technique from a chatbot
Sadly, Thomas, this is not even all that new. I have seen people do searches on the internet for how to do one thing at a time and then cobble together some code that does something but perhaps not quite what they intended. Some things are just inefficient such as reading data from a file, doing some calculations, writing the results to another file, reading them back in and doing more calculations and writing them out again and so on. Yes, there can be value in storing intermediate results but why read it in again when it is already in memory? And, in some cases, why not do multiple steps instead of one at a time and so on. How many people ask how to TEST the code they get, especially from an AI-like ...? -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Wednesday, April 3, 2024 7:51 AM To: python-list@python.org Subject: Re: A technique from a chatbot On 4/3/2024 1:27 AM, AVI GROSS via Python-list wrote: > I am a tad confused by a suggestion that any kind of GOTO variant is bad. The suggestion runs counter to the reality that underneath it all, compiled programs are chock full of GOTO variants even for simple things like IF-ELSE. > > Consider the code here: > >>> def first_word_beginning_with_e( list_ ): >>> for word in list_: >>> if word[ 0 ]== 'e': return word >>> something_to_be_done_at_the_end_of_this_function() > > If instead the function initialized a variable to nothing useful and in the loop if it found a word beginning with e and it still contained nothing useful, copied it into the variable and then allowed the code to complete the loop and finally returned the variable, that would simply be a much less efficient solution to the problem and gain NOTHING. There are many variants you can come up with and when the conditions are complex and many points of immediate return, fine, then it may be dangerous. But a single return is fine. > > The function does have a flaw as it is not clear what it should do if nothing is found. Calling a silly long name does not necessarily return anything. > > Others, like Thomas, have shown other variants including some longer and more complex ways. > > A fairly simple one-liner version, not necessarily efficient, would be to just use a list comprehension that makes a new list of just the ones matching the pattern of starting with an 'e' and then returns the first entry or None. This shows the code and test it: > > text = ["eastern", "Western", "easter"] > > NorEaster = ["North", "West", "orient"] > > def first_word_beginning_with_e( list_ ): >return(result[0] if (result := [word for word in list_ if word[0].lower() == 'e']) else None) > > print(first_word_beginning_with_e( text )) > print(first_word_beginning_with_e( NorEaster )) > > Result of running it on a version of python ay least 3.8 so it supports the walrus operator: > > eastern > None The OP seems to want to return None if a match is not found. If a Python function ends without a return statement, it automatically returns None. So nothing special needs to be done. True, that is probably a special case, but it suggests that the problem posed to the chatbot was not posed well. A truly useful chatbot could have discussed many of the points we've been discussing. That would have made for a good learning experience. Instead the chatbot produced poorly constructed code that caused a bad learning experience. > [snip...] -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: A missing iterator on itertools module?
Antoon,
Even if the suggested solution offers a partial result, you would need
specific requirements to determine what should be done if one or more of the
parts being cycled is shorter than the others. Stopping at that point is one
option. Another is to continue but only interleave ones still producing and
in the same order.
There is a function in itertools called zip_longest() that might be
considered as it keeps going but substitutes a customizable value for
"missing" parts. You could then, perhaps, make a change so that sentinel is
not passed along.
-Original Message-
From: Python-list On
Behalf Of Antoon Pardon via Python-list
Sent: Wednesday, April 3, 2024 5:11 AM
To: python-list@python.org
Subject: Re: A missing iterator on itertools module?
Op 28/03/2024 om 17:45 schreef ast via Python-list:
> Hello
>
> Suppose I have these 3 strings:
>
> s1 = "AZERTY"
> s2 = "QSDFGH"
> s3 = "WXCVBN"
>
> and I need an itertor who delivers
>
> A Q W Z S C E D C ...
>
> I didn't found anything in itertools to do the job.
The documentation mentions a roundrobin recipe.
>
> So I came up with this solution:
>
>
> list(chain.from_iterable(zip("AZERTY", "QSDFGH", "WXCVBN")))
>
> ['A', 'Q', 'W', 'Z', 'S', 'X', 'E', 'D', 'C', 'R', 'F', 'V', 'T', 'G',
> 'B', 'Y', 'H', 'N']
But if your strings are not equal, this will only produce a partial result.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: A technique from a chatbot
I am a tad confused by a suggestion that any kind of GOTO variant is bad. The suggestion runs counter to the reality that underneath it all, compiled programs are chock full of GOTO variants even for simple things like IF-ELSE. Consider the code here: >> def first_word_beginning_with_e( list_ ): >> for word in list_: >> if word[ 0 ]== 'e': return word >> something_to_be_done_at_the_end_of_this_function() If instead the function initialized a variable to nothing useful and in the loop if it found a word beginning with e and it still contained nothing useful, copied it into the variable and then allowed the code to complete the loop and finally returned the variable, that would simply be a much less efficient solution to the problem and gain NOTHING. There are many variants you can come up with and when the conditions are complex and many points of immediate return, fine, then it may be dangerous. But a single return is fine. The function does have a flaw as it is not clear what it should do if nothing is found. Calling a silly long name does not necessarily return anything. Others, like Thomas, have shown other variants including some longer and more complex ways. A fairly simple one-liner version, not necessarily efficient, would be to just use a list comprehension that makes a new list of just the ones matching the pattern of starting with an 'e' and then returns the first entry or None. This shows the code and test it: text = ["eastern", "Western", "easter"] NorEaster = ["North", "West", "orient"] def first_word_beginning_with_e( list_ ): return(result[0] if (result := [word for word in list_ if word[0].lower() == 'e']) else None) print(first_word_beginning_with_e( text )) print(first_word_beginning_with_e( NorEaster )) Result of running it on a version of python ay least 3.8 so it supports the walrus operator: eastern None -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Tuesday, April 2, 2024 3:31 PM To: python-list@python.org Subject: Re: A technique from a chatbot On 4/2/2024 1:47 PM, Piergiorgio Sartor via Python-list wrote: > On 02/04/2024 19.18, Stefan Ram wrote: >>Some people can't believe it when I say that chatbots improve >>my programming productivity. So, here's a technique I learned >>from a chatbot! >>It is a structured "break". "Break" still is a kind of jump, >>you know? >>So, what's a function to return the first word beginning with >>an "e" in a given list, like for example >> [ 'delta', 'epsilon', 'zeta', 'eta', 'theta' ] >> >>? Well it's >> def first_word_beginning_with_e( list_ ): >> for word in list_: >> if word[ 0 ]== 'e': return word >> >>. "return" still can be considered a kind of "goto" statement. >>It can lead to errors: >> >> def first_word_beginning_with_e( list_ ): >> for word in list_: >> if word[ 0 ]== 'e': return word >> something_to_be_done_at_the_end_of_this_function() >>The call sometimes will not be executed here! >>So, "return" is similar to "break" in that regard. >>But in Python we can write: >> def first_word_beginning_with_e( list_ ): >> return next( ( word for word in list_ if word[ 0 ]== 'e' ), None ) > > Doesn't look a smart advice. > >>. No jumps anymore, yet the loop is aborted on the first hit It's worse than "not a smart advice". This code constructs an unnecessary tuple, then picks out its first element and returns that. The something_to_be_done() function may or may not be called. And it's harder to read and understand than necessary. Compare, for example, with this version: def first_word_beginning_with_e(target, wordlist): result = '' for w in wordlist: if w.startswith(target): res = w break do_something_else() return result If do_something_else() is supposed to fire only if the target is not found, then this slight modification will do: def first_word_beginning_with_e(target, wordlist): result = '' for w in wordlist: if w.startswith(target): res = w break else: do_something_else() return result [Using the "target" argument instead of "target[0]" will let you match an initial string instead of just a the first character]. > First of all, I fail to understand why there > should be no jumps any more. > It depends on how "return" and "if" are handled, > I guess, in different context. > Maybe they're just "masked". > In any case, the "compiler" should have just > done the same. > >>(if I guess correctly how its working). > > Second, it is difficult to read, which is bad. > The "guess" above is just evidence of that. > > My personal opinion about these "chatbots", is > that, while they might deliver clever solutions, > they are not explaining *why* these solutions > should be considered "clever". > Which is the most
Re: Multiplication
Is this a April 1 post for fools. Multiplication with an asterisk symbol is built into python. The same symbol used in other contexts has other contexts has an assortment of largely unrelated meanings such as meaning everything when used to import. On Mon, Apr 1, 2024, 1:27 PM Piergiorgio Sartor via Python-list < python-list@python.org> wrote: > On 01/04/2024 10.40, Stefan Ram wrote: > > Q: How can I multiply two variables in Python? I tried: > > > > a = 2 > > b = 3 > > print( ab ) > > > > but it did not work. > > > > A: No, this cannot work. To multiply, you need the multiplication > > operator. You can import the multiplication operator from "math": > > > > Code example: > > > > from math import * > > > > a = 2 > > b = 3 > > print( a * b ) > > I guess the operator "*" can be imported from any module... :-) > > bye, > > -- > > piergiorgio > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
RE: Can you help me with this memoization simple example?
I am not sure if it was made clear that there is a general rule in python for
what is HASHABLE and lists are changeable while tuples are not so the latter
can be hashed as a simple copy of a list, albeit the contents must also be
immutable.
The memorize function uses a dictionary to store things and thus the things are
hashed to decide how to store it in the inner representation of a dictionary
and anything new that you want to look up in the dictionary has similar
considerations as it is hashed to see where in the dictionary to look for it.
Of course, if you add enough overhead and the memorize function you make gets
relatively few requests that are identical, it may not be worthwhile.
-Original Message-
From: Python-list On
Behalf Of MRAB via Python-list
Sent: Sunday, March 31, 2024 3:24 PM
To: python-list@python.org
Subject: Re: Can you help me with this memoization simple example?
On 2024-03-31 09:04, marc nicole wrote:
> Thanks for the first comment which I incorporated
>
> but when you say "You can't use a list as a key, but you can use a
> tuple as a key,
> provided that the elements of the tuple are also immutable."
>
> does it mean the result of sum of the array is not convenient to use
> as key as I do?
> Which tuple I should use to refer to the underlying list value as you
> suggest?
>
I was suggesting using `tuple` on the argument:
def memoize(f):
cache = {}
def g(*args):
key = tuple(args[0]), args[1]
if key not in cache:
cache[key] = f(args[0], args[1])
return cache[key]
return g
> Anything else is good in my code ?
>
> Thanks
>
> Le dim. 31 mars 2024 à 01:44, MRAB via Python-list
> a écrit :
>
> On 2024-03-31 00:09, marc nicole via Python-list wrote:
> > I am creating a memoization example with a function that adds up
> / averages
> > the elements of an array and compares it with the cached ones to
> retrieve
> > them in case they are already stored.
> >
> > In addition, I want to store only if the result of the function
> differs
> > considerably (passes a threshold e.g. 50 below).
> >
> > I created an example using a decorator to do so, the results
> using the
> > decorator is slightly faster than without the memoization which
> is OK, but
> > is the logic of the decorator correct ? anybody can tell me ?
> >
> > My code is attached below:
> >
> >
> >
> > import time
> >
> >
> > def memoize(f):
> > cache = {}
> >
> > def g(*args):
> > if args[1] == "avg":
> > sum_key_arr = sum(list(args[0])) / len(list(args[0]))
>
> 'list' will iterate over args[0] to make a list, and 'sum' will
> iterate
> over that list.
>
> It would be simpler to just let 'sum' iterate over args[0].
>
> > elif args[1] == "sum":
> > sum_key_arr = sum(list(args[0]))
> > if sum_key_arr not in cache:
> > for (
> > key,
> > value,
> > ) in (
> > cache.items()
> > ): # key in dict cannot be an array so I use the
> sum of the
> > array as the key
>
> You can't use a list as a key, but you can use a tuple as a key,
> provided that the elements of the tuple are also immutable.
>
> > if (
> > abs(sum_key_arr - key) <= 50
> > ): # threshold is great here so that all
> values are
> > approximated!
> > # print('approximated')
> > return cache[key]
> > else:
> > # print('not approximated')
> > cache[sum_key_arr] = f(args[0], args[1])
> > return cache[sum_key_arr]
> >
> > return g
> >
> >
> > @memoize
> > def aggregate(dict_list_arr, operation):
> > if operation == "avg":
> > return sum(list(dict_list_arr)) / len(list(dict_list_arr))
> > if operation == "sum":
> > return sum(list(dict_list_arr))
> > return None
> >
> >
> > t = time.time()
> > for i in range(200, 15000):
> > res = aggregate(list(range(i)), "avg")
> >
> > elapsed = time.time() - t
> > print(res)
> > print(elapsed)
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Popping key causes dict derived from object to revert to object
Lori,
The list comprehension you are thinking of does work if you change things a
bit. But it is not a great idea as a main purpose of a dict is that using a
hash means things are found in linear time. A comprehension iterates on all
values. If you wanted to select just some items to keep in a list, your code
could be modified from:
dict_list = [d.pop('a') for d in dict_list]
to have an IF clause that would specify something like comparing it to the
item you do not want to keep.
But your idiom might be better done to make another dictionaly, not list
with something like:
New_dict = {key:value for key in dict if key != "whatever"}
Or variants on that. It builds a new dictionary, at nontrivial expense, as
compared to using del on an existing dictionary.
-Original Message-
From: Python-list On
Behalf Of Loris Bennett via Python-list
Sent: Monday, March 25, 2024 2:56 AM
To: python-list@python.org
Subject: Re: Popping key causes dict derived from object to revert to object
Grant Edwards writes:
> On 2024-03-22, Loris Bennett via Python-list
wrote:
>
>> Yes, I was mistakenly thinking that the popping the element would
>> leave me with the dict minus the popped key-value pair.
>
> It does.
Indeed, but I was thinking in the context of
dict_list = [d.pop('a') for d in dict_list]
and incorrectly expecting to get a list of 'd' without key 'a', instead
of a list of the 'd['a]'.
>> Seem like there is no such function.
>
> Yes, there is. You can do that with either pop or del:
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> d
> {'a': 1, 'b': 2, 'c': 3}
> >>> d.pop('b')
> 2
> >>> d
> {'a': 1, 'c': 3}
>
>
> >>> d = {'a':1, 'b':2, 'c':3}
> >>> del d['b']
> >>> d
> {'a': 1, 'c': 3}
>
> In both cases, you're left with the dict minus the key/value pair.
>
> In the first case, the deleted value printed by the REPL because it
> was returned by the expression "d.pop('b')" (a method call).
>
> In the second case is no value shown by the REPL because "del d['b']"
> is a statement not an expression.
Thanks for pointing out 'del'. My main problem, however, was failing to
realise that the list comprehension is populated by the return value of
the 'pop', not the popped dict.
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Popping key causes dict derived from object to revert to object
I am glad, Lori, you found a solution another way.
Actually, Lori, I think you were right in looking for a built-in method that
complements pop() by returning everything else other than the item mentioned.
There are philosophical and practical considerations that were no doubt
considered and a reality that the functionality did exist albeit not in a
pipelined format.
Consider a language like LISP which sort of processed lists of things in which
a major concept was getting the first item or getting all the rest. Lots of
LISP programs had complex structures using CAR() and CDR() nested various ways
to say extract the third item as CAR(CDR(CDR(X))) to the point where some
commonly used combos became functions with names like CADDR().
There are again many languages where functions or methods are available that
include an exclusion variant from a collection perhaps by doing something as
simple as using a minus sign to indicate what to remove, as is commonly done in
R to remove a column in a data.frame while keeping the order of the remaining
columns the same.
Lots of languages had similar concepts about ordered data structures but
dictionaries may be a bit of something else and initially in Python were not
guaranteed to have any kind of order. Python dicts are more like unordered
sets.
So although there remains a concept of not first/rest but this/rest, I suspect
there was some thought about the process that ended in deciding not to supply
some functionality. When you use pop() on something like a large dictionary,
the original is left intact and is ignored and a copy of a single element is
made and returned. To do the opposite and return the rest has two choices. One
is to make a big copy of the rest of the dictionary and the other is to use del
internally and return the modified dict. The functions you could write do the
latter.
So why not add one or more methods to do that? Who knows? But I think some may
have considered it not needed including some who felt no need for a pipeline
method when del would do. Another consideration was the common idiom for
iterating on a collection. Besides pop() you can get lists of dictionary
entries, keys or values that you can work with and you can even iterate with
"for key in dict: ..."
Given how many ways common things can be done, and given that adding too many
methods has costs including new users not understanding all the nuanced
differences, this fairly useful functionality was left out.
Unfortunately, I think they were wrong here as instead we hear often from
people like you who assume things would work other ways. I still think it would
be simple enough to have had a .removekeys(keys) that would work in a pipeline
to modify the dict by removing one or more items and perhaps another
.removevalues(values) but at some point you may keep adding methods nobody ever
uses. The reality is that many trivial one-liner comprehensions can easily do
many such things using iteration.
But many simply do not work well in pipelined fashion and thus may need to be
embedded in a method of your own by subclassing dict or rolling your own.
-Original Message-
From: Loris Bennett
Sent: Monday, March 25, 2024 2:45 AM
To: avi.e.gr...@gmail.com
Cc: python-list@python.org
Subject: Re: Popping key causes dict derived from object to revert to object
writes:
> Loris wrote:
>
> "Yes, I was mistakenly thinking that the popping the element would leave
> me with the dict minus the popped key-value pair. Seem like there is no
> such function."
>
> Others have tried to explain and pointed out you can del and then use the
> changed dict.
>
> But consider the odd concept of writing your own trivial function.
>
> def remaining(adict, anitem):
> _ = adict.pop(anitem)
> # alternatively duse del on dict and item
> return adict
>
>
remaining({"first": 1, "second": 2, "third": 3}, "second")
> {'first': 1, 'third': 3}
>
>
> Or do you want to be able to call it as in dict.remaining(key) by
> subclassing your own variant of dict and adding a similar method?
No, 'del' does indeed do what I wanted, although I have now decided I
want something else :-) Nevertheless it is good to know that 'del'
exists, so that I don't have to reinvent it.
Cheers,
Loris
--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list
RE: Popping key causes dict derived from object to revert to object
Loris wrote:
"Yes, I was mistakenly thinking that the popping the element would leave
me with the dict minus the popped key-value pair. Seem like there is no
such function."
Others have tried to explain and pointed out you can del and then use the
changed dict.
But consider the odd concept of writing your own trivial function.
def remaining(adict, anitem):
_ = adict.pop(anitem)
# alternatively duse del on dict and item
return adict
>>> remaining({"first": 1, "second": 2, "third": 3}, "second")
{'first': 1, 'third': 3}
Or do you want to be able to call it as in dict.remaining(key) by
subclassing your own variant of dict and adding a similar method?
--
https://mail.python.org/mailman/listinfo/python-list
RE: Configuring an object via a dictionary
If we are bringing up other languages, let's return to what was part of the
original question.
How van a dictionary be used in python if your goal is to sort of use it to
instantiate it into a set of variables and values inside the local or global or
other namespaces? Can we learn anything from other programming languages with
other paradigms?
I was thinking that in one sense, python has other kinds of collections such as
a class or class member with internal variables that also has some features
similar enough.
I mean if I were to call a function with one argument being a dictionary like
{"a":1, "c": 3} where in theory any number of similar key value pairs might
exist or not be present, then the question sounded like a request to create
variables, in this case a, and c, that hold those values. I will not debate the
wisdom of doing that, of course. Nor will I debate what should happen if some
of those variable names are already in use. OK?
There are ways to do this albeit some are a tad obscure such as taking a
printable representation of the dictionary and editing it and feeding that to
an eval.
But could you serve a similar purpose by passing an object containing varying
internal fields (or methods) and values including some of the dataclasses Dave
Neal is highlighting? Is there some overlap with using dictionaries?
In Javascript, I would say they have a very different view in which all kinds
of things overlap and their objects can even implement what others might call
arrays albeit a tad weirder like allowing missing indices and ignoring
non-numeric indices for some purposes. Someone may wish to chime in if people
can and do take such objects passed in and split them into individual variables
as requested.
What I wanted to mention is some philosophical issues in the R language in
which the default language used other data structures to loosely support what
dictionaries often do in python. They have named lists where components
optionally can have a name as in list(a=5, "b"=6, 7, "hello world") and they
have data structures called environments. Loosely, an environment is a set of
name=value pairs and is pretty much like a dictionary and is mostly used behind
the scenes as the interpreter searches for variable names in a sequence of
environments such as the parent environment. But you can use environments all
over the place on purpose and as noted, a named list simulates an environment.
So a fairly common usage when using base R is to take a data.frame (which is at
first approximation a list of named vectors all the same length) and want to
work with the column names without extra typing. If I had a data.frame that
looked like mydf <- data.frame(a=1:3, b=3:1) then if I wanted to add
corresponding entries, I might type:
result <- mydf$a + mydf$b
inside a with statement, an environment is put on the stack consisting of the
contents of mydf and you can now use things like:
result <- with(mydf, a+b)
There is more but the point is for those who hate the extra typing of long
names, this can be useful and a tad dangerous if the variable names are not
unique.
But as R delays evaluation in various ways, a similar style has evolved in an
assortment of packages to the point where I often program in a style that looks
like:
result <-
mydf |>
mutate(newcol = a+b, doubled = 2*newcol, ...)
The point is that all kinds of things that seem like local variables can be
used as if they had been declared withing some environment but that are not
available once you leave that region.
So perhaps there is some validity in a request to be able to just pass an
argument as a dictionary in python and have it unpacked.
In actuality, I wonder if the OP is aware of the unpacking functionality you
can get using **dict in a function invocation.
Say you have a function that expects any combination of three variables called
the unoriginal names of alpha/beta/gamma and you want to call it with a
dictionary that contains any subset of those same keys and nothing else:
mydict = {"alpha":5, "beta":6}
def unpacked(alpha=None, beta=None, gamma=None):
print(alpha if alpha != None else "No alpha")
print(beta if beta != None else "No beta")
print(gamma if gamma != None else "No gamma")
If I now call unpacked with ** mydict:
>>> unpacked(**mydict)
5
6
No gamma
Within the body of that function, arguably, I can tell if something was passed
or not, assuming None or any other sentinel I choose is not used except in
setting the default.
And if you add other unknown names, like delta, they seem to be ignored without
harmful effects or can be caught in other ways.
>>> unpacked({"gamma":7, "delta":8})
{'gamma': 7, 'delta': 8}
No beta
No gamma
So given a function similar to this, and you wanting LOCAL variables set, how
would this do if you wanted these three or any number, and in a row:
mydict = {"beta":6, "alpha":5}
def dict_to_vars(alpha=None, beta=None, gamma=None):
RE: Configuring an object via a dictionary
A part of the Python view of the world is about a concept of whether something is "truthy" or not and thus many corners of the language do not care what kind of object an expression returns. If the object is returned in a context looking for not a Boolean value but a truth value, it is evaluated and in other scenarios, left alone to propagate in the code. Changing such behavior would be a very serious undertaking, and frankly, silly. But if anyone really wants an actual Boolean, then the non-not operator should do the trick as !(whatever) takes what follows as a truthy value and negates it and a second ! brings it back as a True/False as in !!(whatever) And for many data types, perhaps all, you can use the bool() function that I believe follows the same rules about being truthy. Both of the above should be fairly easy to use in any rare contexts that demand a more standard Boolean result as in some other languages. It is one of many strengths of python that supports varieties of polymorphism. And it allows a value to be passed or returned that can both be viewed as some kind of object of many kinds and seen as a Boolean for considerations like flow of control. -Original Message- From: Python-list On Behalf Of Dan Sommers via Python-list Sent: Friday, March 15, 2024 5:33 PM To: python-list@python.org Subject: Re: Configuring an object via a dictionary On 2024-03-15 at 15:48:17 -0400, Thomas Passin via Python-list wrote: > [...] And I suppose there is always the possibility that sometime in > the future an "or" clause like that will be changed to return a > Boolean, which one would expect anyway. Not only is the current value is way more useful, but changing it would be a compatibility and maintenance nightmare. If I want Java, I know where to find it. :-) -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Extract lines from file, add to new files
Dave, You and I have had some experience in teaching or tutoring others and I think it fair to say our motivation is closer to teaching someone how they can fish for themselves rather than just handing them a fully-cooked fish. My favorite kinds of questions, thus, include someone who explains what they are trying to do and shows some code along with indications of what it produced (including error messages) and what it should produce. Then the question should not be a request to just FIX THIS or WRITE IT FOR ME but asking if someone can show what they did wrong with some idea where it went wrong. This may not be so common but it allows faster and easier help. But others out there probably can be helped including people who want suggestions of approaches to try. We have discussed some issues here and elsewhere where some purists tend to want to stick with close to basic aspects of the language. In some R communities, some get upset if someone asks a question (or others supply an answer) using non-official add-ons called the tidyverse and I suspect some here don't really like discussions that focus mainly on numpy/pandas or so many other modules. Yet, the reality is, that except for students who are supposed to be learning the basics of the language and show solutions compatible with what is being taught in class, others can and should be open to simple solutions using non-standard tools that are widely available and vetted. If a student is asked to use some method to calculate a value like pi, then suggesting they use math.pi is not useful. But if the assignment is to do something in trigonometry, it seems a good thing to use as a part of a larger solution rather than either embedding a constant to N decimal places or calculating it on the f ly. I think people like us who volunteer to reply should consider our choices too. I think it is fair to reply, as I saw on the tutor forum, that the code shown uses a method that is not the way the replier would do it but nonetheless offer some thoughts on particular python coding errors. I am talking about someone who wanted to implement a binary tree using a numpy array or something with minimal explanation and demanded a fix. They never supplied enough info and I think the few who replied backed away. As you mention, they seemed resistant to starting over and using data structures that are perhaps more amenable, and in any case, the code shown seemed mostly focused on how to make random and non-redundant names. I will end with a comment. I have heard of interview techniques for a job where they deliberately supply a problem in which the goal is not so much to be easy to solve in front of them in real time but to watch how the person looking for a job responds to the uncertainties and asks follow-up questions or verbalizes things like, if it is like this, I might use this technique but if you also need that then ... So, I shudder to think what happens if someone being interviewed turns around and asks us and further confuses things with changes to make it harder to recognize they are asking for outside help. The answer expected may well be to NOT use say the older versions of PASCAL to do something but switch to something better suited (and for that matter available.) I would not want to program the DES encryption/decryption method in Pascal again! And these days, it seems much better to just find a module or package that meets such needs. Avi -Original Message- From: Python-list On Behalf Of dn via Python-list Sent: Saturday, February 3, 2024 5:02 PM To: python-list@python.org Subject: Re: Extract lines from file, add to new files Every trainer, in any field, has to deal with these problems - all the time, and over-and-over. On 4/02/24 06:58, Thomas Passin via Python-list wrote: > In my view this whole thread became murky and complicated because the OP > did not write down the requirements for the program. Requirements are > needed to communicate with other people. An individual may not need to > actually write down the requirements - depending on their complexity - > but they always exist even if only vaguely in a person's mind. The > requirements may include what tools or languages the person wants to use > and why. > > If you are asking for help, you need to communicate the requirements to > the people you are asking for help from. > > The OP may have thought the original post(s) contained enough of the > requirements but as we know by now, they didn't. There is another possible interpretation in such situations (not necessarily this one): that the person is fixated on a particular solution (and unable/unwilling to adjust his/her thinking to consider more widely). Thus, the question is not: 'here's an entire problem, how can it be solved', but more: 'I have a solution, and want help to implement it (and only it) just-so'. The latter is an interesting psychology: 1 an
RE: Extract lines from file, add to new files
We substantially agree with that, Thomas. In the best of all possible worlds, someone who gets stuck will sit down and try to carefully spell out things in ways like you mention and, incidentally, may often catch the error or figure out how to do it and not even send in a request! LOL! I think a main thing that the OP and others can do is to not just be abstract but supply a small example including what output they expect and perhaps what they did not receive properly along with error messages. Rich had not tried doing what he wanted in python, yet. I don't know if he did any other parts yet. I think he was still in a somewhat abstract design state and hoping for someone to push him at something like a resource to continue and he did accept suggestions on what now seem somewhat random things to read as people guessed. In a sense, I and others can take some blame for the way we widened the problem while trying to look at it our own way. But looking back, I think Rich asked close to what he wanted. An example might have helped such as: 'I have information about multiple clients including an email address and a human name such as: u...@domain.com Sally f...@bark.com Barky I want to save the info in a file or maybe two in such a way that when I write a program and ask it to send an email to "f...@bark.com" then it finds an entry that matches the email address and then uses that to find the matching name and sends a specified message with a specific salutation in front like "Dear Barky,". I was thinking of having two files with one having email address after another and the other having the corresponding names. I want to use python to search one file and then return the name in the other. Are there other and perhaps better ways commonly used to associate one keyword with a value?' Back to me. The above is not polished but might still have engendered a discussion such as how to keep the files synchronized. Yes, the OP could have added endless clauses saying that they are not asking how to create the data files or keep them synchronized but just how to match them. The reply in this case possibly could have suggested they count the lines they have read until the match and, assuming no blank lines are used, read a second file till the nth line. We might have been done quickly and THEN had a long discussion about other ways! I have participated, like you, in another forum designed for tutoring and I think the rules and expectations there may be a bit different. Over here, it is fairer to expect people to take a bit of time and ask clearer questions. We live and we learn and then Alzheimer's ... -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Saturday, February 3, 2024 12:59 PM To: python-list@python.org Subject: Re: Extract lines from file, add to new files In my view this whole thread became murky and complicated because the OP did not write down the requirements for the program. Requirements are needed to communicate with other people. An individual may not need to actually write down the requirements - depending on their complexity - but they always exist even if only vaguely in a person's mind. The requirements may include what tools or languages the person wants to use and why. If you are asking for help, you need to communicate the requirements to the people you are asking for help from. The OP may have thought the original post(s) contained enough of the requirements but as we know by now, they didn't. The person asking for help may not realize they don't know enough to write down all the requirements; an effort to do so may bring that lack to visibility. Mailing lists like these have a drawback that it's hard to impossible for someone not involved in a thread to learn anything general from it. We can write over and over again to please state clearly what you want to do and where the sticking points are, but newcomers post new questions without ever reading these pleas. Then good-hearted people who want to be helpful end up spending a lot of time trying to guess what is actually being asked for, and maybe never find out with enough clarity. Others take a guess and then spend time working up a solution that may or may not be on target. So please! before posting a request for help, write down the requirements as best you can figure them out, and then make sure that they are expressed such that the readers can understand. On 2/3/2024 11:33 AM, avi.e.gr...@gmail.com wrote: > Thomas, > > I have been thinking about the concept of being stingy with information as > this is a fairly common occurrence when people ask for help. They often ask > for what they think they want while people like us keep asking why they want > that and perhaps offer guidance on how to get closer to what they NEED or a > better way. > > In retrospect, Rich did give all the info he thought he needed. It boiled > down to saying that he wants to
RE: Extract lines from file, add to new files
Thomas, I have been thinking about the concept of being stingy with information as this is a fairly common occurrence when people ask for help. They often ask for what they think they want while people like us keep asking why they want that and perhaps offer guidance on how to get closer to what they NEED or a better way. In retrospect, Rich did give all the info he thought he needed. It boiled down to saying that he wants to distribute data into two files in such a way that finding an item in file A then lets him find the corresponding item in file B. He was not worried about how to make the files or what to do with the info afterward. He had those covered and was missing what he considered a central piece. And, it seems he programs in multiple languages and environments as needed and is not exactly a newbie. He just wanted a way to implement his overall design. We threw many solutions and ideas at him but some of us (like me) also got frustrated as some ideas were not received due to one objection or another that had not been mentioned earlier when it was not seen as important. I particularly notice a disconnect some of us had. Was this supposed to be a search that read only as much as needed to find something and stopped reading, or a sort of filter that returned zero or more matches and went to the end, or perhaps something that read entire files and swallowed them into data structures in memory and then searched and found corresponding entries, or maybe something else? All the above approaches could work but some designs not so much. For example, some files are too large. We, as programmers, often consciously or unconsciously look at many factors to try to zoom in on what approaches me might use. To be given minimal amounts of info can be frustrating. We worry about making a silly design. But the OP may want something minimal and not worry as long as it is fairly easy to program and works. We could have suggested something very simple like: Open both files A and B In a loop get a line from each. If the line from A is a match, do something with the current line from B. If you are getting only one, exit the loop. Or, if willing, we could have suggested any other file format, such as a CSV, in which the algorithm is similar but different as in: Open file A Read a line in a loop Split it in parts If the party of the first part matches something, use the party of the second part Or, of course, suggest they read the entire file, into a list of lines or a data.frame and use some tools that search all of it and produce results. I find I personally now often lean toward the latter approach but ages ago when memory and CPU were considerations and maybe garbage collection was not automatic, ... -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Wednesday, January 31, 2024 7:25 AM To: python-list@python.org Subject: Re: Extract lines from file, add to new files On 1/30/2024 11:25 PM, avi.e.gr...@gmail.com wrote: > Thomas, on some points we may see it differently. I'm mostly going by what the OP originally asked for back on Jan 11. He's been too stingy with information since then to be worth spending much time on, IMHO. > Some formats can be done simply but are maybe better done in somewhat > standard ways. > > Some of what the OP has is already tables in a database and that can > trivially be exported into a CSV file or other formats like your TSV file > and more. They can also import from there. As I mentioned, many spreadsheets > and all kinds of statistical programs tend to support some formats making it > quite flexible. > > Python has all kinds of functionality, such as in the pandas module, to read > in a CSV or write it out. And once you have the data structure in memory, al > kinds of queries and changes can be made fairly straightforwardly. As one > example, Rich has mentioned wanting finer control in selecting who gets some > version of the email based on concepts like market segmentation. He already > may have info like the STATE (as in Arizona) in his database. He might at > some point enlarge his schema so each entry is placed in one or more > categories and thus his CSV, once imported, can do the usual tasks of > selecting various rows and columns or doing joins or whatever. > > Mind you, another architecture could place quite a bit of work completely on > the back end and he could send SQL queries to the database from python and > get back his results into python which would then make the email messages > and pass them on to other functionality to deliver. This would remove any > need for files and just rely on the DB. > > There as as usual, too many choices and not necessarily one best answer. Of > course if this was a major product that would be heavily used, sure, you > could tweak and optimize. As it is, Rich is getting a chance to improve his > python skills no matter which way he goes. > > > > -Original Message- >
RE: Extract lines from file, add to new files
This discussion has circled back to where it started. It illustrates quite a few points about how many different ways someone can do something as well as doing it using different tools and also about how others may see aspects of mission creep as they look for making it perfect when it need not be. I mean the OP wants a simple personal tool and is not currently concerned with lots of things. The data seems to be in a data base and can be extracted from there into one or two files mostly for convenience. Presumably, all the usual additions/deletions/modifications are being done in the DB. If so, then as long as his latest vision of having two synchronized files is done consistently by making both, they are in sync. In addition, it sounds like the original idea was a tad too simple. The OP wanted to add a single-line or so at the head of a message that simply interpolates a name. This could be handled with all kinds of form-letter-merge methods as well as search and replace methods of great generality. But, for now, the OP is asking for something truly simple and would be happy with a solution that is slow and clunky and uses a few utilities in a shell script on LINUX. I took a look at doing this on bash and ran into some surprises as this would have been trivial for me in a shell like ksh. Setting variables is now done in a subshell which was done why? All I know is if you could simply read a file one line at a time in a loop and set one or more variables, then it becomes easy to use something like combining an echo and a cat of the original message to make a message with a variable heading and feed that to the mailer. And if the storage format was CSV, again, you could easily peel it apart using something like cut or awk or whatever. The main reason for trying it in python may have been frustration with doing it as a shell script. Fair enough. But the idea of two files may have been made in the first place by the original idea of such a script. The question posed was sort of how to search in one and then somehow find the corresponding part of another. The mindset of the OP may have been focused in a direction we were not clear on. As an example, there are ways to search file A and get back a line number of the first match and then using perhaps another utility you ask it to scan file B and just print out that line number and nothing else. The current solution the OP is considering sounds like a loop that goes through both files in parallel. This would mean when one loop finds what it is looking for, the other delivers what is in the same location. That may work well enough for the purpose. But I have learned over the years that having a few well-designed tools and ideas and using them consistently, is often a good way to get results even if it is not as imaginative and efficient as some other. If you are allowed to design something from top to bottom, there may be very nice solutions. But if asked to just do one small part in a design that already feels it has solved other aspects, then suggesting they start over is often not taken well. LOL! I happen to think some storage format such as CSV would be a good idea IFF doing this in Python for some reasons I mentioned earlier. I am not in love with CSV and use other data formats as needed and some others would be fine as long as your toolkit included some easy and reliable way to read the contents in and manipulate to get your results. I do programming in multiple languages and whether I am using python or something like R, I often end up importing the data into some data.frame format, performing a pipeline of manipulations using it and perhaps other such structures and generating results and optionally saving some back to files. Tons of well-tested and comfortable tools can be used if the data is set up right so even when it can be easily done by hand, it may not be worth the effort as the standard tools and tricks work and work fast enough. Rich (AKA the OP) seems satisfied with having a solution. It may not be ideal but for his needs, ... -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Wednesday, January 31, 2024 10:36 AM To: python-list@python.org Subject: Re: Extract lines from file, add to new files On 1/31/2024 9:05 AM, Rich Shepard via Python-list wrote: > On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote: > >> If I had a script that's been working for 30 years, I'd probably just use >> Python to do the personalizing and let the rest of the bash script do the >> rest, like it always has. The Python program would pipe or send the >> personalized messages to the rest of the bash program. Something in that >> ballpark, anyway. > > Thomas, > > A bash shell script looks easier for me and more promising. Using a while > loop (one for the name file the other for the address file), and sed for > putting the name at the head of the message replacing a generic placeholder > should work with the
RE: Extract lines from file, add to new files
Thomas, on some points we may see it differently.
Some formats can be done simply but are maybe better done in somewhat
standard ways.
Some of what the OP has is already tables in a database and that can
trivially be exported into a CSV file or other formats like your TSV file
and more. They can also import from there. As I mentioned, many spreadsheets
and all kinds of statistical programs tend to support some formats making it
quite flexible.
Python has all kinds of functionality, such as in the pandas module, to read
in a CSV or write it out. And once you have the data structure in memory, al
kinds of queries and changes can be made fairly straightforwardly. As one
example, Rich has mentioned wanting finer control in selecting who gets some
version of the email based on concepts like market segmentation. He already
may have info like the STATE (as in Arizona) in his database. He might at
some point enlarge his schema so each entry is placed in one or more
categories and thus his CSV, once imported, can do the usual tasks of
selecting various rows and columns or doing joins or whatever.
Mind you, another architecture could place quite a bit of work completely on
the back end and he could send SQL queries to the database from python and
get back his results into python which would then make the email messages
and pass them on to other functionality to deliver. This would remove any
need for files and just rely on the DB.
There as as usual, too many choices and not necessarily one best answer. Of
course if this was a major product that would be heavily used, sure, you
could tweak and optimize. As it is, Rich is getting a chance to improve his
python skills no matter which way he goes.
-Original Message-
From: Python-list On
Behalf Of Thomas Passin via Python-list
Sent: Tuesday, January 30, 2024 10:37 PM
To: python-list@python.org
Subject: Re: Extract lines from file, add to new files
On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>
>> Fine, my toy example will still be applicable. But, you know, you haven't
>> told us enough to give you help. Do you want to replace text from values
>> in a file? That's been covered. Do you want to send the messages using
>> those libraries? You haven't said what you don't know how to do.
>> Something
>> else? What is it that you want to do that you don't know how?
>
> Thomas,
>
> For 30 years I've used a bash script using mailx to send messages to a
list
> of recipients. They have no salutation to personalize each one. Since I
> want
> to add that personalized salutation I decided to write a python script to
> replace the bash script.
>
> I have collected 11 docs explaining the smtplib and email modules and
> providing example scripts to apply them to send multiple individual
> messages
> with salutations and attachments.
If I had a script that's been working for 30 years, I'd probably just
use Python to do the personalizing and let the rest of the bash script
do the rest, like it always has. The Python program would pipe or send
the personalized messages to the rest of the bash program. Something in
that ballpark, anyway.
> Today I'm going to be reading these. They each recommend using .csv input
> files for names and addresses. My first search is learning whether I can
> write a single .csv file such as:
> "name1","address1"
> "mane2","address2"
> which I believe will work; and by inserting at the top of the message
block
> Hi, {yourname}
> the name in the .csv file will replace the bracketed place holder
If the file contents are going to be people's names and email addresses,
I would just tab separate them and split each line on the tab. Names
aren't going to include tabs so that would be safe. Email addresses
might theoretically include a tab inside a quoted name but that would be
extremely obscure and unlikely. No need for CSV, it would just add
complexity.
data = f.readlines()
for d in data:
name, addr = line.split('\t') if line.strip() else ('', '')
> Still much to learn and the batch of downloaded PDF files should educate
> me.
>
> Regards,
>
> Rich
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Aw: Re: Extract lines from file, add to new files
Rich, You may want to broaden your perspective a bit when people make suggestions. Karsten did not spell out a full design and should not need to. But consider this as a scenario. You want to send (almost) the same message to one or more recipients. So call a program, perhaps some variant on a shell script, that does some prep work such as maybe creating a temporary or working directory/folder. Had one copy of your message ready in a file somewhere, Have a way to get a list of recipients intended and the file or files containing enough info to link email addresses to human names and anything else such as their preferred pronoun or address. Now have the script call your super-duper python program with enough info so it can find the folder to put amended COPIES of your letter into as well as. Perhaps the email address intended in the filename or whatever works for you. Your program will then simply identify each email recipient you want and look up the other info and prepend the customized salutation, or make substitutions in the template and write out a new file in the designated folder with perhaps the email address as the filename. When your loop ends, exit the python program with success, or perhaps report some failure. The shell script now resumes by checking the exit status and if OK, continuing to enter the folder and loop on all file contests and invoke the functionality to send each copied/enhanced file to the intended recipient. If you also need to support attachments, you can figure out how to attach the same ones to each as I assume those are not changed for each recipient. It may keep track of how many worked or failed and eventually clear out the files and perhaps the folder and you are done. This is NOT a required way to do it but for what sounds like a limited personal project, it should work well enough and have you do the limited amount of work you need in Python. Having said that, you can likely also easily do everything without python and I have written some huge shell scripts in my time to do way more complex things. But learning how to do things like this well in python can be time well spent as long as you don't tackle too much at a time and get overwhelmed. -Original Message- From: Python-list On Behalf Of Rich Shepard via Python-list Sent: Tuesday, January 30, 2024 12:53 PM To: python-list@python.org Subject: Re: Aw: Re: Extract lines from file, add to new files On Tue, 30 Jan 2024, Karsten Hilbert wrote: > Why not foxus on just the part you think you are better off using python, > namely personalization ? > > Create personalized files and send them with your trusted mailx solution ? Karsten, Too much time. And while mailx accepts the '-a' option for attachments but has none for individual salutations. Regards, Rich -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Extract lines from file, add to new files
I deleted the contents of the message so I can avoid both of the deadly sins of top posting and bottom posting and chance committing the sin of replying without any context. Of course, I am only replying to Jon wishing a real or feigned good luck to the OP. But seriously, the OP, AKA Rich, is making clear that he is making a tool for his own use. It sounds like he wants to maintain a data repository of his own with some info about his clients and then have the ability to specify a name and pop up an email directed to them, or something along those lines. Without further info, this sounds like what quite a few popular and even free mailers already do to some extent as you have an associated address book. Some start doing possible matches as you type. I hazard Rich is not using something this simple as he does seem to have some form-letter merge or similar functionality in mind, such as automating the seemingly mandatory "Dear XXX" salutation. But if he is the creator and maintainer of his client data and chooses not to use one of many available applications, then it seems he already has reasons he wants a particular design for the data and simply wants us to help him use it the way he wants. That could be communicated a bit more clearly but after many messages back and forth, I have not exactly understood it. In my opinion, having created all kinds of mailers over the years, sometimes the hard way, This strikes me as not really being about what mailing functionality exists at all. As a general rule, you first create supporting parts for your mail then pass them along to functionality that assembles the mail as a set of headers and a body and dispatches it. You as the programmer need to supply a body, tell it who to put on TO/CC/BB lines, perhaps provide a subject, perhaps specify attachments, and off you go. The exact methods will differ. But what Rich presumably needs to do is have his program interact with him in specifying who he wants to mail to and then looking it up in whatever file arrangement it contains. If he also wants to use other parts such as a human name or address inside the body of the text, his program needs to merge the text he supplies along with extracted parts of the data. Or is that not what he wants? The above could be quite straightforward and I recall doing things like this with a simple shell script in UNIX. Specifically, I created a text file where I recorded info for each person in some format like NAME|PHONE|EMAIL|COMMENT Then to search for someone, you could use something like grep to find a name by anchoring to the beginning or ending with the "|" so it does not match the text in another field such as email or address, and use other utilities ranging from cut to awk and getting the parts you want into variables and then interpolate them into a message template and so on. Of course, doing it in python is a good way to go too and should not be hard once it is decided how to store the data. But again, this is re-inventing things that others have already done. The python modules include many ways to store modular data. The books I have read mention them all the time. Pick one. And, yes, you can choose to maintain two files if that design works for you. Consider some storage method that stores data in sections like: [NAME one] First: whatever Email: whatever [NAME two] First: ... Email: ... There are specific formats along these lines and you can get python modules that you ask for "NAME one" and it reads the file until it finds a section as "[NAME one]" or not. If found, it returns the variables/values right below it. So your two step algorithm may consist of two files with one file containing just names, perhaps to use a grep functionality on. Some of those names will have a matching section like the above somewhere in the other file. So if you want to send mail to "Jo" then your program may search for all names starting with "Jo" and offer you "John Smith" and perhaps also "Joachim Martillo". The program takes whichever full name(s) you then select and calls a function that uses that full name to search the second file to find an exact match and returns what it finds there such as an email address. But unless you have lots of contacts, as already discussed, there are far easier ways to do things in a more brute force way. Take a one-line per entry format such as a CSV or TSV and extract whatever column contains the name as needed to do the first search. Yes, this tends to mean reading the entire file. And, for the record, I am not a fan of hiding replies at the bottom except for short messages. I prefer to use some combination of in-line if addressing many points in the original and mainly the top with perhaps a preface explaining what is being addressed. The reader is usually capable of digging below if they want to know more. But this is a more religious war having nothing to do with python specifically. My frustration is that I often want to help
RE: Extract lines from file, add to new files
It can be quite frustrating figuring out what someone wants, Grant, especially when they just change it. It is worse when instead of starting a new thread with an appropriate subject line, it continues and old one that was also frustrating to understand. It sounds though like another attempt to do something perhaps a different way. Both attempts seem to be to use some form of storage of a set of email addresses plus other info like a name that can be used to make a customized email. Frankly, this should have been fairly easy to do without so much back and forth. I don't care how the email is actually sent, but the rest could have been done any number of ways such as storing the data as rows in a CSV file or saved using JSON format and so on. It was never made clear why two files were needed and then somehow linked and searched. If the goal is to be able to search for something like a name and THEN find an email address, that seems quite trivial if they are I the same file in some format. If the number of items is small, reading it all in should not be a big deal and you can use a regular expression or other method to locate the entry you want and extract the additional info. If you have lots of data, reading line after line may be less useful than just using a database and a query. One way to stop feeling frustrated is to stop reading the thread. -Original Message- From: Python-list On Behalf Of Grant Edwards via Python-list Sent: Monday, January 29, 2024 5:54 PM To: python-list@python.org Subject: Re: Extract lines from file, add to new files On 2024-01-29, Rich Shepard via Python-list wrote: > On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote: > >> No, I hadn't ... but I am reading it now. > > Perhaps I missed the answer to my question when reading the io module. It > explains how to open/write/read files of text and binary data, not passing > a variable's value from one file to a place-keeper in another file. It's not at all clear (to me) what you're asking about. When you talk about "files" are you referring to data files? Python modules within a single program? Seperate Python programs? Something else? The phrase "place-keeper in another file" sounds a bit like you're trying to do templating. There are many, many ways to do templating in Python -- ranging from literal 'f-strings' to powerful templating engines that are used to construct entire web sites: https://www.google.com/search?q=python+templating https://docs.python.org/3/tutorial/inputoutput.html#tut-f-strings https://en.wikipedia.org/wiki/Jinja_(template_engine) -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Extract lines from file, add to new files
Rich, You got an overly general reply to a question many of us may not understand. You have not hinted at how the two files are organized, perhaps with an example. There are several general solutions that may apply. Some involve reading in both files into data structures and perhaps linking them together in some way such as a data.frame or binary tree. You can then process individual request in memory/ The second should be straightforward as long as text is text. If the first file tells you to search for XYZ then you search the second file for XYZ and read in whatever is associated with it and do your thing. Without a bit more specific detail, you may not get more than a suggestion as to how to read in files. -Original Message- From: Python-list On Behalf Of Rich Shepard via Python-list Sent: Monday, January 29, 2024 12:38 PM To: python-list@python.org Subject: RE: Extract lines from file, add to new files On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote: > No, I hadn't ... but I am reading it now. Perhaps I missed the answer to my question when reading the io module. It explains how to open/write/read files of text and binary data, not passing a variable's value from one file to a place-keeper in another file. I'll keep searching for a solution. Rich -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: extend behaviour of assignment operator
Guenther, It is best not to suggest a drastic fix for a more limited problem. As a general rule, many programming languages only have a pointer concept even vaguely along the lines you want for garbage collection purposes. An area of memory may have stored alongside it how many other things point at it but not which ones. As long as it is decremented when a pointer leaves, it works. If you want to design objects that can store additional info when invoked properly, go for it. No change to python would be needed. In your example, you could create an object initialized by cube([10,1,1], "a") which now might remember that something called "a" once pointed at it. But you then have to figure out how to ensure than when "a" is deleted or reset or goes out of the current environment, that things are properly updated. I am not so sure how easy it would be to change the language so it pays attention to what it is giving a pointer too and then goes and tells ... -Original Message- From: Python-list On Behalf Of Guenther Sohler via Python-list Sent: Tuesday, January 9, 2024 2:15 AM To: python-list@python.org Subject: extend behaviour of assignment operator Hi, when i run this code a = cube([10,1,1]) b = a i'd like to extend the behaviour of the assignment operator a shall not only contain the cube, but the cube shall also know which variable name it was assigned to, lately. I'd like to use that for improved user interaction. effective code should be: a=cube([10,1,1]) a.name='a' b=a b.name='b' # i am aware that a.name also changes can decorators also be used with assignment operators ? thank you for your hints -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Extract lines from file, add to new files
Whoa, Олег Сивоконь! I do not understand any arguments about whether comments are, or are not an object. From one perspective, python comments have even less meaning than whitespace and simply do not exist. I mean once a naked "#" is seen, the rest of that line is effectively discarded by the interpreter. The supposed multi-line comments though, meaning something like a triple set of quotes up to another set are probably objects in the sense that they create a text literal in a context where there is no usual pointer to them and are thus largely ignored and perhaps garbage collected. Some programs look for them in code as part of documentation and it is possible they are stored in some object that "holds" aspects of a function. But I think that talking about some inconsistency in python because comments are not an object is a bit silly. Why should they be? If I choose to leave lots of blank lines between my function definitions or statements, do they need to be made into an object, or ignored as superfluous but legal whitespace? I have seen many debates about some form of purity and generally, they get boring. Nobody in their right mind can defend any computer language as being 100% right for every purpose. Python has some reasonable tradeoffs and is highly popular and there are other languages with other tradeoffs you can use instead. At this point, making changes without disrupting things gets ever harder. -Original Message- From: Python-list On Behalf Of Left Right via Python-list Sent: Sunday, January 14, 2024 7:28 AM To: Chris Angelico Cc: python-list@python.org Subject: Re: Extract lines from file, add to new files > Second time to ameliorate wording-dispute in this thread! The original > phrase was: "[modified] BNF". Some of us have worked with various forms > and evolutions of BNF since back in the days of COBOL-60 proposals, and > know it when we see it! OK, here are the conceptual differences between what Python grammar language does and what you'd expect from anything that's based on BNF, modified or not: Python isn't a context-free language, so the grammar that is used to describe it doesn't actually describe the language... so, it's a "pretend grammar" that ignores indentation. BNF is supposed to be used to describe the language, it's not a "pretend" or "pseudo" grammar, in a way we have at least two established grammar for pseudo-code. BNF and derivatives don't have an inherent mechanism for tiebreaks. The mechanism is necessary because BNF rules can be tried in any order. Some grammar languages derived from BNF declare ambiguous grammars invalid, some allow ambiguity, but say that the longest prefix wins, and if there's still ambiguity after that, then such grammar is invalid, some have special constructs to define "priority" etc. My reading of Python grammar is that it works like PEG, where rules are tried in the order they are defined. This makes it less expressive, but easier to work with. This is, probably, the most fundamental difference between the BNF family and the PEG family. BNF and family languages rarely incorporate elements of Perl-like regular expression parsing in the language (i.e. things like lookaheads, lookbehinds etc.) This is more typical of the PEG family. On top of this, the Python grammar language has a bunch of "inventions" that are unique to it (I've never seen any other grammar language use '.' in the same way Python uses it). So, there's that too. Having worked with a bunch of different grammar languages, the one used for Python isn't a recognizable BNF derivative. I think the authors used this as a description in the same way as today a lot of programmers would use the word "IDE" to describe any text editor or "REST" to describe any kind of server-client protocol over HTTP and so on. Or, how we'd use "Xerox" to name a copier machine, even if that company didn't manufacture it, and even if the tech used for copying is completely different. And that's why I wrote that the grammar is actually more like PEG, adding that it's neither, but seems to fall more into that later category. > Yes it is hard to read - and even harder to learn-from; This wasn't my point. My point is that it's hard to learn languages that are "one off" in the group languages that all share a similar set of rules. The difficulty comes from the surprise caused by the unique use, not because there's something inherently difficult about reading grammar languages. In fact, however you look at Python's grammar language, in a sense, it's a lot easier to read than Python itself because it has significantly fewer rules. Of course, the number of rules doesn't entirely capture the difficulty, but it's a useful metric. > In Python, everything is an object. As long as the LHS is a legal-object > which makes sense for the situation, it can be used. This is a very interesting statement... I don't think you are fully aware of what it might mean :) Here are just a few
RE: Extract lines from file, add to new files
Straight Ahead, on average,
I am not sure what your beef is as apparently it is always something else than
some others assumed.
If your point is that you want consistency, sure that would be nice. But maybe
part of the inconsistency I am not sure you mean is an artifact of the language.
There are programming languages where a keyword like "def" is used to create a
function and similarly some where variables are declared some special way like
"let" or "var" and "const" or "class" to provide the right hints. Some
languages skip that and just let you pop up a variable as in "x=5" both
declares and instantiates a variable. Strictly speaking, in a language like
python, everything is an object and some of the way of creating them remains
bound to other ideas like declaring a class uses key words as does declaring a
function.
But consider a language like R which declares a function similar to anything
else:
X <- 5
Times2 <- function(x) { x*2 }
The keyword has been moved as compared to the python:
Def Times2(x):
return(x*2)
It is not a question of being better or worse, but simply speaking, there is no
special keyword to start the process before the name, but rather a keyword
later that is really similar to a lambda method. Secondarily, not relying on
indentation, may make it easier in some cases to insert the function
declaration anywhere a statement might fit as there is no special keyword.
Other languages have their own chosen ways and many will fail various tests you
can come up with.
Could python have chosen some other way that would have fit some grammar needs?
Probably. But it did not.
If you want to play a common grammar game, you can define a higher-level
construct I will simply call A that is decomposed into two or more subcontexts
one of which is B and then populate the tree on each side with more and more
specific stuff. Then when talking about what is allowed in context alpha, where
all expressions are allowed, say it supports A or anything in the tree below
it. In another context, say it only supports B and anything below that.
Think of it a bit like subclassing.
Perhaps you can then build a complex description that can be instantiated by
code to define all valid programs from invalid ones. But it would remain a pain
to implement all the tools you want to help you, including in an editor or
programming environment, where figuring out the context may become very
difficult.
Still, making things very loose and general so that your design looks simple
has many negative tradeoffs too, including allowing rather nonsensical things.
People who create programming languages have various goals in mind that guide
what they choose. Python was not made to be a mathematically perfect object
that guided being able to have programs proven to work and so on. It is
acknowledged various aspects do not please some people or others and I am not
defending it.
I am wondering if what is being discussed is in any way a serious issue.
The original question in this thread really was a minor one and how it became
whatever this is, well, I give up! LOL!
-Original Message-
From: Left Right
Sent: Sunday, January 14, 2024 4:15 PM
To: avi.e.gr...@gmail.com
Cc: Chris Angelico ; python-list@python.org
Subject: Re: Extract lines from file, add to new files
> You said function. I made a function. You said "head of a for loop
> clause". I put it there. Problem was underspecified.
I also wrote a lot of letters, if you combine them very liberally,
without any regard to the order in which they were written or the
context in which they were used, you may come up with very surprising
findings.
> But if you're trying to tell me that a def statement should be a valid
> assignment target,
Why not just read what I wrote and work from there? No, I didn't
write anything even remotely similar to this... I don't want function
definition to be an assignment target. I was giving an example of how
Python grammar works, how the rules govern what can or cannot be used
in a particular place...
In other words, if you aren't sure you understand the question, why
are you trying to reply to it? Is your goal to learn the meaning of
the question by giving arbitrary replies and hoping that the author of
the question restates it so that you understand it? If so, I believe,
the better strategy would be to simply ask to restate the question.
Will save you the round-trip.
> You provided a way to create an anonymous function and that was not enough.
> I wonder if you could throw in the new := walrus operator to similarly make
> a named lambda function in a similar way.
The person you are replying to didn't understand the question and has
written something irrelevant. It's not about being "enough". I
honestly don't know why they are spending so much energy replying to
my messages :|
> Python grew and there was regular pressure to add keywords which might break
> existing programs. So, yes,
RE: Extract lines from file, add to new files
Chris, It gets frustrating when people demand too much when strictly speaking, it is not really needed or may cause new problems. How often do you really think anyone out there NEEDS to define a function in the context mentioned? You provided a way to create an anonymous function and that was not enough. I wonder if you could throw in the new := walrus operator to similarly make a named lambda function in a similar way. And, in any case, you can make a function factory that returns a function and call that function with appropriate arguments that might then create a wide variety of functions by doing the def within. Languages are tools and need not be the end-all for all people. You do not want the language to get so complex that programmers do not have any idea what valid code does. Consider the mess in multiple inheritance and trying to figure out which of many classes will be the one where a method is finally called using some diamond algorithm. It is both extremely powerful but also silly to overuse such features. Avi -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Sunday, January 14, 2024 8:34 AM To: python-list@python.org Subject: Re: Extract lines from file, add to new files On Mon, 15 Jan 2024 at 00:27, Left Right wrote: > > > What do you mean? > > > > for x in lambda: ...: > > ... > > > > Perfectly grammatical. > > 1. You put the lambda definition in the wrong place (it should be in > the left-hand side, or as Python calls it "star_targets", but you put > it into "star_expressions", which would be where the right-hand side > is drawn from). > 2. You used what Python calls "lambdadef" in place of what Python > calls "function_def". I.e. lambda definition and function definition > are two different things, at least as far as grammar is considered. > > So, you solved a different problem. You said function. I made a function. You said "head of a for loop clause". I put it there. Problem was underspecified. But if you're trying to tell me that a def statement should be a valid assignment target, I don't know what you're smoking, but I want you to keep it a long way away from me. Can you name ANY language in which that would make the slightest bit of sense? ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Extract lines from file, add to new files
It can be worth considering why a language is designed or altered in certain ways to see if there was a tradeoff that made it seem worthwhile or easier than some other choice. Python grew and there was regular pressure to add keywords which might break existing programs. So, yes, sometimes, a keyword was re-used in a different context. And, yes, it was not originally conceived in a purely object oriented context. If you wanted to start over and built a new language very similar to python, you might indeed make other choices now that seem more seamlessly to fit together. You could set aside and reserve hundreds of keywords or some way to extend keywords by insisting anything staring with "key_" cannot be used in a variable name. You might design all the main objects supported to all support a function that provides a length as well as every other method needed so it looks purely object oriented. But perhaps that would make it a tad harder to program it using other ways. As an example, I can ask some sort program to order the results by the length of items by passing it the function that does lengths as an argument. If instead all we had was a method, that might be a bit different and perhaps someone would simply make a tiny function that when called, invoked the method. So, we have a hybrid of sorts and have to live with it, warts and all, and some of the warts may be seen by some as beauty marks. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Sunday, January 14, 2024 7:32 AM To: python-list@python.org Subject: Re: Extract lines from file, add to new files On Sun, 14 Jan 2024 at 23:28, Left Right wrote: > Having worked with a bunch of different grammar languages, the one > used for Python isn't a recognizable BNF derivative. That might possibly be because it isn't? It's not BNF. It's PEG. Or are you a long way behind the times? > For example, you may say "functions in Python are > objects", but you cannot put a function definition in the head of the > for loop clause. What do you mean? for x in lambda: ...: ... Perfectly grammatical. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Extract lines from file, add to new files
If the data in the input file is exactly as described and consists of alternating lines containing a name and email address, or perhaps an optional blank line, then many solutions are possible using many tools including python programs. But is the solution a good one for some purpose? The two output files may end up being out of sync for all kinds of reasons. One of many "errors" can happen if multiple lines in a row do not have an "@" or a person's name does, for example. What if someone supplied more than one email address with a comma separator? This may not be expected but could cause problems. Some of the other tools mentioned would not care and produce garbage. Grep as an example could be run twice asking for lines with an "@" and then lines without. In this case, that would be trivial. Blank lines, or ones with just whitespace, might need another pass to be omitted. But a real challenge would be to parse the file in a language like Python and find all VALID stretches in the data and construct a data structure containing either a valid name or something specific like "ANONYMOUS" alongside an email address. These may be written out as soon as it is considered valid, or collected in something like a list. You can do further processing if you want the results in some order or remove duplicates or bad email addresses and so on. In that scenario, the two files would be written out at the end. Python can do the above while some of the other tools mentioned are not really designed for it. Further, many of the tools are not generally available everywhere. Another question is why it makes sense to produce two output files to contain the data that may not be linked and would not be easy to edit and keep synchronized such as to remove or add entries. There are many ways to save the data that might be more robust for many purposes. It looks like the application intended is a sort of form letter merge where individual emails will be sent that contain a personalized greeting. Unless that application has already been written, there are many other ways that make sense. One obvious one is to save the data in a databases as columns in a table. Other ones are to write one file with entries easily parsed out such as: NAME: name | EMAIL: email Whatever the exact design, receiving software could parse that out as needed by the simpler act of reading one line at a time. And, of course, there are endless storage formats such as a CSV file or serializing your list of objects to a file so that the next program can load them in and operate from memory on all the ones it wants. The two file solution may seem simpler but harks back to how some computing was done in early days when list of objects might be handled by having multiple arrays with each containing one aspect of the object and updating required rememebreing to touch each array the same way.. That can still be a useful technique when some operations being done in a vectoried manner might be faster than an array of objects, but is more often a sign of poor code. -Original Message- From: Python-list On Behalf Of Grizzy Adams via Python-list Sent: Friday, January 12, 2024 1:59 AM To: Rich Shepard via Python-list ; Rich Shepard Subject: Re: Extract lines from file, add to new files Thursday, January 11, 2024 at 10:44, Rich Shepard via Python-list wrote: Re: Extract lines from file, add to (at least in part) >On Thu, 11 Jan 2024, MRAB via Python-list wrote: >> From the look of it: >> 1. If the line is empty, ignore it. >> 2. If the line contains "@", it's an email address. >> 3. Otherwise, it's a name. If that is it all? a simple Grep would do (and save on the blank line) -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: A problem with str VS int.
Roel, I sent a similar reply in private to someone who may not be listening as their mind is made up. This points to a serious problem with people not testing hypotheses adequately. Perhaps for homework, we can assign a request for a Python program that creates a random sample of quite a few digit strings of lengths from say 1 to 5 and compares then to each other as strings and then as integer representations and prints out whether the two methods match or not. Perhaps that might get them to discard the hypothesis and be a bity more open. I still have had to deal with people who want to know why "two" is more than "three" and truly do not understand that just because a human sees "two" as a number, that does not mean anything about another human in whose language it may be "zwei" let alone a computer program not expecting character strings to mean anything unless programmed to examine them a certain way. And often, the same people cannot sole a simple puzzle like "SEND" + "MORE" == "MONEY" -Original Message- From: Python-list On Behalf Of Roel Schroeven via Python-list Sent: Tuesday, December 12, 2023 3:58 AM To: python-list@python.org Subject: Re: A problem with str VS int. Op 12/12/2023 om 9:22 schreef Steve GS via Python-list: > With all these suggestions on > how to fix it, no one seems to > answer why it fails only when > entering a two-digit number. > One and three work fine when > comparing with str values. It > is interesting that the > leading 0 on a two digit > worked. Still, one digit and > three digit work but not two. Three-digit numbers work because you're comparing to another three-digit numbers. When two integer numbers have the same number of digits, their lexicographical ordering matches their numeric ordering. One-digit numbers don't work fine: >>> "5" < "400" False even though we can construct cases where it seems as if they do: >>> "1" < "400" True Two-digit numbers sometimes seem to work: >>> "30" < "400" True But other times clearly don't work: >>> "50" < "400" False String comparison first looks at the first characters of both operands. If they are different (as in the examples above), their ordering is used regardless of all the other characters that come after, and regardless of the length of the string. Try working through some examples (make sure to pick examples with a wide variety of first digits) and you'll see why it sometimes seems to work, but very unreliably. -- "In the old days, writers used to sit in front of a typewriter and stare out of the window. Nowadays, because of the marvels of convergent technology, the thing you type on and the window you stare out of are now the same thing.” -- Douglas Adams -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: A problem with str VS int.
Steve,
I would say converting to a number, as you eventually did, is the way to go.
When you compare character strings, it will not be in numeric order. Compare
"80" with "400" and since 8 is greater than 4, the comparison is over and
"80" is greater then "40" even though 80 is way less than 400.
The only way to get the kind of comparison you want would be to pad with
zeroes so all your "numbers" are the same length. In that case, "080" would
indeed test as less than "400" but rarely does that seem a good idea. If the
user can enter any text, they might enter ".01" or "hello" or al kinds of
nonsense.
If you converted to numbers and tested whether it failed, ...
-Original Message-
From: Python-list On
Behalf Of Steve GS via Python-list
Sent: Saturday, December 9, 2023 9:42 PM
To: python-list@python.org
Subject: A problem with str VS int.
If I enter a one-digit input or a three-digit number, the code works but if
I enter a two digit number, the if statement fails and the else condition
prevails.
tsReading = input(" Enter the " + Brand + " test strip reading: ")
if tsReading == "": tsReading = "0"
print(tsReading)
if ((tsReading < "400") and (tsReading >= "0")):
tsDose = GetDose(sReading)
print(tsReading + "-" + tsDose)
ValueFailed = False
else:
print("Enter valid sensor test strip Reading.")
I converted the variable to int along with the if statement comparison and
it works as expected.
See if it fails for you...
Steve
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Newline (NuBe Question)
Dave, I gave an example, again, and make no deep claims so your comments may be valid, without any argument. I mentioned CSV and a related family such as TSV as they were a common and simple data format that has long been used. There are oodles of others and yes, these days many people can read directly from formats like some from EXCEL. But for data that can be shared to almost anyone using anything, something like Comma Separated Values is often used. And some programs that generate such data simply keep appending a line at a time to a file and do not have any header line. There are even some programs that may not tolerate a file with a header line, or comments or other optional things, and some where header lines you can create would cause problems such as using an extended character set or escaped characters. I have worked with these files in many languages and environments and my thought process here focused on recent work in R, albeit much applies everywhere. My point was really not about CSV but the convenience and advantages of data structures you can access by name when you want and sometimes also by position when you want. Too many errors can happen when humans doing programming are not able to concentrate. It is similar to arguments about file names. In the old UNIX days, and the same for other systems like VMS, a filename tended to have a format where relatively few characters were allowed and it might have two parts with the latter being an extension of up to 3 characters, or whatever. So file names like A321G12.dat were common and also next to it similar unpronounceable other file names. It was easy to confuse them and even people who worked with them regularly would forget what it might mean or use the wrong one. Well, if I load in a CSV in a language like R and there is no header line, as with some other data structures, it may make up a placeholder set of names like V1, V2 and so on. Yes, there are ways to specify the names as they are read in or afterward and they can be changed. But I have seen lots of CSV files offered with way too many columns and no names as well as documentation suggesting what names can be added if you wish. This may be a bit off topic, but I want to add a bit in this context about additional concepts regarding name. As mentioned, there is a whole set of add-ons people sometimes use and in R, I like the tidyverse family and it allows some fairly sophisticated things to be done using names. There are ways to specify you want a subset of a data.frame (sometimes a version called a tibble) and you can ask for say all columns starting with "xyz" or containing it or ending with it. That can be very helpful if say we wave columns containing the height and weight and other metrics of say people in three clinics and your column names embed the name of the clinic, or other such examples, and you want to select one grouping for processing. You cannot easily do that without external info is it is just positional. An extension of this is how compactly you can do fairly complex things such as asking to create lots of new columns using calculations. You can specify, as above, which sets of columns to do this too and that you want the results for each XYY in XYZ.mean and XYZ.std and so on. You can skip oodles of carefully crafted and nested loops because of the ability to manipulate using column names at a high and often abstract level. And, just FYI, many other structures such as lists in R also support names for components. It can be very useful. But the overall paradigm compared to Python has major differences and I see strengths and weaknesses and tradeoffs. Your dictionary example is one of them as numpy/pandas often make good use of them as part of dealing with similar data.frame type structures that are often simpler or easier to code with. There is lots of AI discussion these days and some of what you say is applicable in that additional info besides names might be useful in the storage format to make processing it more useful. That is available in formats related to XML where fairly arbitrary markup can be made available. Have to head out as this is already long enough. -Original Message- From: 'DL Neil' Sent: Monday, November 27, 2023 2:49 AM To: avi.e.gr...@gmail.com; python-list@python.org Subject: Re: Newline (NuBe Question) Avi, On 11/27/2023 4:15 PM, avi.e.gr...@gmail.com wrote: > Dave, > > Back on a hopefully more serious note, I want to make a bit of an analogy > with what happens when you save data in a format like a .CSV file. > > Often you have a choice of including a header line giving names to the > resulting columns, or not. > > If you read in the data to some structure, often to some variation I would > loosely call a data.frame or perhaps something like a matrix, then without > headers you have to specify what you want positionally or create your own > names for columns to use.
RE: Newline (NuBe Question)
Dave, Back on a hopefully more serious note, I want to make a bit of an analogy with what happens when you save data in a format like a .CSV file. Often you have a choice of including a header line giving names to the resulting columns, or not. If you read in the data to some structure, often to some variation I would loosely call a data.frame or perhaps something like a matrix, then without headers you have to specify what you want positionally or create your own names for columns to use. If names are already there, your program can manipulate things by using the names and if they are well chosen, with no studs among them, the resulting code can be quite readable. More importantly, if the data being read changes and includes additional columns or in a different order, your original program may run fine as long as the names of the columns you care about remain the same. Positional programs can be positioned to fail in quite subtle ways if the positions no longer apply. As I see it, many situations where some aspects are variable are not ideal for naming. A dictionary is an example that is useful when you have no idea how many items with unknown keys may be present. You can iterate over the names that are there, or use techniques that detect and deal with keys from your list that are not present. Not using names/keys here might involve a longer list with lots of empty slots to designate missing items, This clearly is not great when the data present is sparse or when the number of items is not known in advance or cannot be maintained in the right order. There are many other situations with assorted tradeoffs and to insist on using lists/tuples exclusively would be silly but at the same time, if you are using a list to hold the real and imaginary parts of a complex number, or the X/Y[/Z] coordinates of a point where the order is almost universally accepted, then maybe it is not worth using a data structure more complex or derived as the use may be obvious. I do recall odd methods sometimes used way back when I programmed in C/C++ or similar languages when some method was used to declare small constants like: #define FIRSTNAME 1 #define LASTNAME 2 Or concepts like "const GPA = 3" And so on, so code asking for student_record[LASTNAME] would be a tad more readable and if the order of entries somehow were different, just redefine the constant. In some sense, some of the data structures we are discussing, under the hood, actually may do something very similar as they remap the name to a small integer offset. Others may do much more or be slower but often add value in other ways. A full-blown class may not just encapsulate the names of components of an object but verify the validity of the contents or do logging or any number of other things. Using a list or tuple does nothing else. So if you need nothing else, they are often suitable and sometimes even preferable. -Original Message- From: Python-list On Behalf Of DL Neil via Python-list Sent: Sunday, November 26, 2023 5:19 PM To: python-list@python.org Subject: Re: Newline (NuBe Question) On 11/27/2023 10:04 AM, Peter J. Holzer via Python-list wrote: > On 2023-11-25 08:32:24 -0600, Michael F. Stemper via Python-list wrote: >> On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote: >>> Of course, for serious work, some might suggest avoiding constructs like a >>> list of lists and switch to using modules and data structures [...] >> >> Those who would recommend that approach do not appear to include Mr. >> Rossum, who said: >>Avoid overengineering data structures. >^^^ > > The key point here is *over*engineering. Don't make things more > complicated than they need to be. But also don't make them simpler than > necessary. > >>Tuples are better than objects (try namedtuple too though). > > If Guido thought that tuples would always be better than objects, then > Python wouldn't have objects. Why would he add such a complicated > feature to the language if he thought it was useless? > > The (unspoken?) context here is "if tuples are sufficient, then ..." At recent PUG-meetings I've listened to a colleague asking questions and conducting research on Python data-structures*, eg lists-of-lists cf lists-of-tuples, etc, etc. The "etc, etc" goes on for some time! Respecting the effort, even as it becomes boringly-detailed, am encouraging him to publish his findings. * sadly, he is resistant to OOP and included only a cursory look at custom-objects, and early in the process. His 'new thinking' has been to look at in-core databases and the speed-ups SQL (or other) might offer... However, his motivation came from a particular application, and to create a naming-system so that he could distinguish a list-of-lists structure from some other tabular abstraction. The latter enables the code to change data-format to speed the next process, without the coder losing-track of the data-type/format. The trouble is,
RE: Newline (NuBe Question)
Isn't it fascinating that a meaningless piece of code used to illustrate something can be analyzed as if it was full of malicious content? Yes, my choice of names was as expected. The numbers chosen had no special meaning other than choosing one number in each of three equivalence classes. But, if you want me to add subtle meaning for generations to examine as it it were a literary work, I offer this: Peter and Paul were studs who got Mary'd. Can we now go back to our regularly scheduled talking about aspects of a computer language? P.S. And just for history, Paul was really Noel Paul Stookey but Peter, Paul & Mary sounded more like new testament characters and I think Noel signifies a birth to Peter and Mary, sort of, which might have fit too unless it was a computer program where a name with an umlaut was once not common. Another interpretation is that Noel came from the Latin word for news. Be that as it may, and I have no interest in this topic, in the future I may use the ever popular names of Primus, Secundus and Tertius and get blamed for using Latin. -Original Message- From: Python-list On Behalf Of DL Neil via Python-list Sent: Sunday, November 26, 2023 4:58 PM To: python-list@python.org Subject: Re: Newline (NuBe Question) On 11/27/2023 12:48 AM, Chris Angelico via Python-list wrote: > On Sun, 26 Nov 2023 at 21:08, Michael F. Stemper via Python-list > wrote: >> >> On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote: >>> Grizz[l]y, >>> >>> I think the point is not about a sorted list or sorting in general It is >>> about reasons why maintaining a data structure such as a list in a program >>> can be useful beyond printing things once. There are many possible examples >>> such as having a list of lists containing a record where the third item is a >>> GPA for the student and writing a little list comprehension that selects a >>> smaller list containing only students who are Magna Cum Laude or Summa Cum >>> Laude. >>> >>> studs = [ >>> ["Peter", 82, 3.53], >>> ["Paul", 77, 2.83], >>> ["Mary", 103, 3.82] >>> ] >> >> I've seen Mary, and she didn't look like a "stud" to me. >> > > That's what happens when you abbreviate "student" though :) Don't > worry, there's far FAR worse around the place, and juvenile brains > will always find things to snigger at, usually in mathematical > libraries with "cumulative" functions. The OP used an abbreviation: "studs". Why? Too lazy to type the full word? Abbreviation has full-meaning in the (narrow) domain? Was wanting something funny, or to snigger over? Was the respondent sniggering? Perhaps he, like the OP, was also saving typing-time by making a joke, hoping that the OP would see the implicit-error in expecting others to understand that "studs" meant "students"? Actually, Peter, Paul, and Mary were a band (https://www.peterpaulandmary.com/), so "studs" is even less expressive when the data also tells a story... Working with "trainees", I avoid the word "student" even though some might see them as synonyms. In my mind, the abbreviation did not readily expand to the full word (mea culpa). Accordingly, would not pass Code Review! For the want of a few characters... (https://en.wikipedia.org/wiki/For_Want_of_a_Nail) -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: RE: Newline (NuBe Question)
Just FYI, I deliberately chose that abbreviation for a sort of irony as for some people college is about almost anything except learning and some people think they are studs and just party and ... And I am very tired of gender discussions. Lots of words now include two or even more genders. Women are often now "actors", not actresses. I see no reason women cannot be studs! But I learn from criticism. If I ever write a program like that and do not feel like typing, will this do? dents = [ ...] Or will that not include students who happen to be edentulous? -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Sunday, November 26, 2023 6:49 AM To: python-list@python.org Subject: Re: RE: Newline (NuBe Question) On Sun, 26 Nov 2023 at 21:08, Michael F. Stemper via Python-list wrote: > > On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote: > > Grizz[l]y, > > > > I think the point is not about a sorted list or sorting in general It is > > about reasons why maintaining a data structure such as a list in a program > > can be useful beyond printing things once. There are many possible examples > > such as having a list of lists containing a record where the third item is a > > GPA for the student and writing a little list comprehension that selects a > > smaller list containing only students who are Magna Cum Laude or Summa Cum > > Laude. > > > > studs = [ > >["Peter", 82, 3.53], > >["Paul", 77, 2.83], > >["Mary", 103, 3.82] > > ] > > I've seen Mary, and she didn't look like a "stud" to me. > That's what happens when you abbreviate "student" though :) Don't worry, there's far FAR worse around the place, and juvenile brains will always find things to snigger at, usually in mathematical libraries with "cumulative" functions. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: RE: Newline (NuBe Question)
That is an entirely different discussion, Michael. I do not know what ideas Guido had ages ago and where he might stand now and I actually seriously disagree with the snippet you quoted below. Python was started long ago as a way to improve in some ways on what was there before. Some of the ideas were nice but also for some purposes, way too slow. If you regard the original versions of LISP, they too simplicity to an extreme and pretty much the main or even only data structure was a list. Functions like CAR and CDR accessed an element but a complex structure resulted in people creating functions with names like CAR and CADADADR to automate climbing a tree of sorts to get to the parts you want. It was a recipe for complexity and errors. My point was that although a list can do so many things in principle, it is not really optimized to do some things that can be way easier using add-ons or your own data structures like objects and he notes the collection library and deque as an example that he is not as much of a purist as you may think. My point was that if you have a fairly detailed and complex application that will manipulate lots of data, then instead of reading in a CSV with many columns and rows recorded into a list of lists, it may make sense to import numpy and pandas that come with all kinds of functionality built in so you do not need to re-invent everything. Just how easy is it using lists of lists to rearrange the order of columns of data, or add new columns containing calculations built from existing columns and so on? Of course, for many purposes, it is indeed overkill albeit once you learn a method, ... I think part of the design of Python, and this is just my guess, included going away from overly specific things done in earlier compiled languages and making it more abstract and inclusive. Arrays or vectors or other such names would normally require everything to be of the same data type and with a fixed length. The list data structure loosened this up quite a bit and also allowed lists within lists. That is great and for some purposes, not very efficient and especially not when your data actually is all of the same type or of fixed length. You can make matrix-like data structures of any depth using lists and it may be hard to traverse such as when you want to multiply two such 2-D matrices. Place the same data (all say floating point numbers) in a vector-like structure that also has stored info about the dimensions, and a simple mathematical calculation accesses any item such as may_tricks[5,42] in the same amount of time as an offset from the top. I have seen this phenomenon in many languages where a somewhat clean and sparse design gets added to, often by others, until some core features are used less often. An example would be R which does have lists nut they are just one form of vectors which are really more the core data idea. It also contains data.frames in the core which are implemented as a list of vectors and more recently a bit more. It was designed to do statistical tasks as one of the main objectives. Yet the graphics functions have been added to so there are by now quite a few independent ways to make graphics using different paradigms. Python also has something like that. And completely new paradigms such as piping data in a chain were added in packages and it became so popular that a version has been added to the core language. Now although the core language includes lots of the functionality you might see in numpy/pandas and you can do all kinds of things, some others kept creating new ways to do things including different data structures that either dealt with weaknesses found or were ore efficient and so on and an entire growing body of alternate ways to do things with lots more power and often more speed that I prefer. A collection of lots of these alternative packages has been assembled and I and others often simply start programs by including the "tidyverse" and the resulting programs might as well be written in a different language to anyone who only knows base R. But I do not see that as a bad thing albeit someone trying to get a part of a program from an AI-like service may need to specify or they may get code they cannot trivially read and evaluate but that works fine once they have loaded the packages. Guido is like many others who create or invent and do it in the way they are proud of. Why change it? But the reality is that first attempts are often done with lots of room for change and improvement. Now if you are teaching a course on Python basics, it may be a good idea to teach the basics and require students to only use in their homework what has already been taught. But if you get a job where the norm is to use modules like numpy, it makes sense to use the expanded language if it results in faster writing perhaps of faster code with fewer mistakes. -Original Message- From: Python-list On Behalf Of Michael F. Stemper via
RE: Newline (NuBe Question)
Grizz[l]y, I think the point is not about a sorted list or sorting in general It is about reasons why maintaining a data structure such as a list in a program can be useful beyond printing things once. There are many possible examples such as having a list of lists containing a record where the third item is a GPA for the student and writing a little list comprehension that selects a smaller list containing only students who are Magna Cum Laude or Summa Cum Laude. studs = [ ["Peter", 82, 3.53], ["Paul", 77, 2.83], ["Mary", 103, 3.82] ] magna = [stud for stud in studs if stud[2] >= 3.5 ] summa = [stud for stud in studs if stud[2] >= 3.75 ] print(studs, magna, summa, sep="\n") OUTPUT: >>> print(studs, magna, summa, sep="\n") [['Peter', 82, 3.53], ['Paul', 77, 2.83], ['Mary', 103, 3.82]] [['Peter', 82, 3.53], ['Mary', 103, 3.82]] [['Mary', 103, 3.82]] Of course, for serious work, some might suggest avoiding constructs like a list of lists and switch to using modules and data structures that are often more efficient to represent your data such as some form of matrix or data.frame. And, yes, you can sort something like the above by name or GPA or number of credits taken but the point was responding to why bother making a list just to print it. The answer is that many and even most programs do a bit more than that and a good choice of data structure facilitates ... -Original Message- From: Python-list On Behalf Of Grizzy Adams via Python-list Sent: Thursday, November 16, 2023 8:41 AM To: python-list@python.org Subject: Re: Newline (NuBe Question) Thursday, November 16, 2023 at 7:47, Thomas Passin via Python-list wrote: Re: Newline (NuBe Question) (at least in part) >I wrote that you don't need the "students" list, which is correct. But >there could be a use for a list. It would let you change the order in >which students appear in the printed output. Knowing how to do that is >a useful skill. But that should be left for a later lesson, not mixed >in here. I have a vague memory of seeing sorted list somewhere ;->) -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Code improvement question
Many features like regular expressions can be mini languages that are designed
to be very powerful while also a tad cryptic to anyone not familiar.
But consider an alternative in some languages that may use some complex set of
nested function calls that each have names like match_white_space(2, 5) and
even if some are set up to be sort of readable, they can be a pain. Quite a few
problems can be solved nicely with a single regular expression or several in a
row with each one being fairly simple. Sometimes you can do parts using some of
the usual text manipulation functions built-in or in a module for either speed
or to simplify things so that the RE part is simpler and easier to follow.
And, as noted, Python allows ways to include comments in RE or ways to specify
extensions such as PERL-style and so on. Adding enough comments above or within
the code can help remind people or point to a reference and just explaining in
English (or the language of your choice that hopefully others later can
understand) can be helpful. You can spell out in whatever level of detail what
you expect your data to look like and what you want to match or extract and
then the RE may be easier to follow.
Of course the endless extensions added due to things like supporting UNICODE
have made some RE much harder to create or understand and sometimes the result
may not even be what you expected if something strange happens like the symbols
①❹⓸
The above might match digits and maybe be interpreted at some point as 12
dozen, which may even be appropriate but a bit of a surprise perhaps.
-Original Message-
From: Python-list On
Behalf Of Peter J. Holzer via Python-list
Sent: Friday, November 17, 2023 6:18 AM
To: python-list@python.org
Subject: Re: Code improvement question
On 2023-11-16 11:34:16 +1300, Rimu Atkinson via Python-list wrote:
> > > Why don't you use re.findall?
> > >
> > > re.findall(r'\b[0-9]{2,7}-[0-9]{2}-[0-9]{2}\b', txt)
> >
> > I think I can see what you did there but it won't make sense to me - or
> > whoever looks at the code - in future.
> >
> > That answers your specific question. However, I am in awe of people who
> > can just "do" regular expressions and I thank you very much for what
> > would have been a monumental effort had I tried it.
>
> I feel the same way about regex. If I can find a way to write something
> without regex I very much prefer to as regex usually adds complexity and
> hurts readability.
I find "straight" regexps very easy to write. There are only a handful
of constructs which are all very simple and you just string them
together. But then I've used regexps for 30+ years, so of course they
feel natural to me.
(Reading regexps may be a bit harder, exactly because they are to
simple: There is no abstraction, so a complicated pattern results in a
long regexp.)
There are some extensions to regexps which are conceptually harder, like
lookahead and lookbehind or nested contexts in Perl. I may need the
manual for those (especially because they are new(ish) and every
language uses a different syntax for them) or avoid them altogether.
Oh, and Python (just like Perl) allows you to embed whitespace and
comments into Regexps, which helps readability a lot if you have to
write long regexps.
> You might find https://regex101.com/ to be useful for testing your regex.
> You can enter in sample data and see if it matches.
>
> If I understood what your regex was trying to do I might be able to suggest
> some python to do the same thing. Is it just removing numbers from text?
Not "removing" them (as I understood it), but extracting them (i.e. find
and collect them).
> > > re.findall(r'\b[0-9]{2,7}-[0-9]{2}-[0-9]{2}\b', txt)
\b - a word boundary.
[0-9]{2,7} - 2 to 7 digits
- - a hyphen-minus
[0-9]{2} - exactly 2 digits
- - a hyphen-minus
[0-9]{2} - exactly 2 digits
\b - a word boundary.
Seems quite straightforward to me. I'll be impressed if you can write
that in Python in a way which is easier to read.
hp
--
_ | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| | | h...@hjp.at |-- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
--
https://mail.python.org/mailman/listinfo/python-list
RE: xor operator (DEPRECATED)
Dom, I hear you. As you say, writing your own extension in something like C++ may not appeal to you even if it is faster. I was wondering if using a generator or something similar in R might make sense. I mean what happens if you write a function that includes a "yield" or two and does a part of what you want. It maintains some internal state between invocations. So you can call it once to setup things then call it repeatedly to keep processing the next item. You stop calling it when you get a result you want, such as that it has seen what you want N times. Since the code stays in memory, it may effectively run faster than some other kinds of functions calls. It can keep things in internal storage such as not just how many N you want but how many it has seen. Your outer function can maintain a list of the items you want to XOR or generate a new one dynamically as needed. It can use functional programming techniques to create a new customized version of the iterator, such as with a value of N built in. You would then call the outer function and let it use the inner function till the result is available or until the data in the iterator runs out or perhaps other tweaks involving two way communication of sorts between the functions. I am NOT suggesting this approach is optimal or fast but merely wondering if something along these lines is worth trying that might speed things up even if not very fast. Such approaches can be even more effective if what you are working on need not all be instantiated up front but can be dynamically calculated or incrementally read from files. With care, you can make multiple instantiations that each iterate over their own sets of data without interference. Just a thought. In a sense, this can be a slightly decent substitute for the non-standard evaluation in R where you can arrange for lots of your data to not be interpreted till absolutely needed. -Original Message- From: Dom Grigonis Sent: Monday, November 13, 2023 10:12 PM To: avi.e.gr...@gmail.com Cc: Grant Edwards ; Python Subject: Re: xor operator (DEPRECATED) Fair point. However, I gave it a shot for the following reason: I couldn’t find a way to make such performant function. Using python builtin components still ends up several times slower than builtin `all`. Cython or numba or similar is not an option as they do not support `truth` values. Or if they do, it ends up slower than pure python variant. So all what is left is writing a proper extension. Which I would prefer not to do for 1 function. I thought maybe `xor`, as in logical XOR functionality in its vanilla case could be compelling. And after doing a bit of search I see that very very few languages have that and it seems for a good reason. Some that do: R, thats all I could find. Although some (if not many) went through the proposal phase. And yes, none of them have a function that I am proposing. So yes, you are right, not a good proposal. But there still seems to be the need for short-circuiting performant implementations in python space. The issue is that there are many variants of what might be needed while there is no efficient solution to sourcing predicates from python to lower level implementations. Someone mentioned that numpy experimented with such implementations in C, but they did not get anywhere with it. The best I could come up with is cached numba for numpy problems, which does perform very well and more than worth it if function is re-used. It even ends up faster than cython or cffi extensions, however can’t have many of those due to JIT and AOT is currently being deprecated (which wouldn’t solve anything anyway). However, as I mentioned earlier it does not apply to this case. So it’s either: a) Something very clever and flexible implemented that covers most of such needs and doesn’t require predicates. b) I welcome any thoughts on this. DG > On 14 Nov 2023, at 04:27, AVI GROSS via Python-list > wrote: > > I was going to ask a dumb question. Has any other language you know of made > something available that does what is being asked for and included it in the > main program environment rather than an add-on? > > A secondary mention here has been whether short-circuiting functions like > "any" and "all" have been augmented with something like "has_n" that > evaluates arguments till it has found n or perhaps n+1 of what it wants then > skips the rest. Does any language supply something like that? What would > such a function return and does it have an "any" or an "all" side? > > It sounds like if I asked if a list of integers has at least n prime numbers > in "any" mode, it should ignore any that are not primes till it finds n > primes or fails and returns true or false. If in "all" mode, I assume it > would have to be the first n items without
RE: xor operator
I was going to ask a dumb question. Has any other language you know of made something available that does what is being asked for and included it in the main program environment rather than an add-on? A secondary mention here has been whether short-circuiting functions like "any" and "all" have been augmented with something like "has_n" that evaluates arguments till it has found n or perhaps n+1 of what it wants then skips the rest. Does any language supply something like that? What would such a function return and does it have an "any" or an "all" side? It sounds like if I asked if a list of integers has at least n prime numbers in "any" mode, it should ignore any that are not primes till it finds n primes or fails and returns true or false. If in "all" mode, I assume it would have to be the first n items without a failure. Fine, but then someone may want to know WHERE you stopped or for you to return the sublist of the ones that made the match, or even return everything that was skipped so you can later process that. Consider a long list of jurors you process to place a dozen that qualify on a jury and then later you want to choose from among the rest for another jury. Human minds can come up with an amazing number of ideas including for "useful" functions or features but I find the vast majority would rarely be used as nobody remembers it is available and some fairly simple method using other functions can easily be cobbled together. -Original Message- From: Python-list On Behalf Of Grant Edwards via Python-list Sent: Monday, November 13, 2023 8:19 PM To: python-list@python.org Subject: Re: xor operator On 2023-11-14, Dom Grigonis via Python-list wrote: > >> Except the 'any' and 'all' builtins are _exactly_ the same as bitwise >> or and and applided to many bits. To do something "in line" with that >> using the 'xor' operator would return True for an odd number of True >> values and False for an even Number of True values. > > Fair point. > > Have you ever encountered the need for xor for many bits (the one > that I am NOT referring to)? Would be interested in what sort of > case it could be useful. Yes, it's used all the time in low-level communications protocols, where it's often implemented in hardware. But, it is also not at all unusual to implement it in software. It's also not that unusual for the "count-ones" part of the function you're asking for to be implemented in hardware by a CPU having an instruction that counts the number of 1 bits in a register. GCC has a low-level builtins called __builtin_popcount() and __builtin-popcountl() that counts the number of 1's in an unsigned (long) int. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Text messages have taken a nasty turn and especially now that so many people have unlimited messages per month in their plan. People overuse them to the point where I opt out of some things like my home town notifications as they bombard me with other things I am not interested in. A major offender now is various forms of security which insist on not letting you into a web site for your bank or other resources unless they first send you an email or text message or perhaps a voice call with random digits to use as verification. Try sitting somewhere with your phone muted as all the beeps get annoying. There are many reasons for preferences and I know many people find it harder to reply to texts on their phone than to emails on a PC with a full keyboard. But all of this is not really here or there. We are talking more about user interface design than about programming, let alone about Python. What strikes me as a useful direction is for people to suggest what resources and methods in the Python world are helpful. Examples would be modules that have been tested and used that do things well such as validating phone numbers or emails, perhaps flexibly so that if a validation fails, they prompt the user asking if they are sure it is correct and maybe offer to let them type it in again for verification. Other ideas as stated recently are routines that don't just ask for a number but specify the purpose, and perhaps messages about what circumstances would trigger a use of the number, such as if fraud is detected, and get you to opt in or refuse. Reusable libraries of sorts, or good documentation of examples, would perhaps help make User Interface design and customer satisfaction better and show Python as a good way to do some kinds of programs. In that light, I wonder if it makes sense to NOT insist people give you their email address at all, and make it optional so they do not need to provide you with something bogus just to go on. -Original Message- From: Python-list On Behalf Of D'Arcy Cain via Python-list Sent: Tuesday, November 7, 2023 11:24 AM To: python-list@python.org Subject: Re: Checking if email is valid On 2023-11-07 08:40, Grant Edwards via Python-list wrote: > If you, as a web developer, want the user to enter a text-message > capable phone number, then ASK FOR THAT! And you may as well ask if they even want you to send texts whether they can technically receive them or not. -- D'Arcy J.M. Cain Vybe Networks Inc. http://www.VybeNetworks.com/ IM:da...@vex.net VoIP: sip:da...@vybenetworks.com -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Python Golf
Discussions like this feel a bit silly after a while. How long something is
to type on a command line is not a major issue and brevity can lead to being
hard to remember too especially using obscure references.
Consider that the Perl version as shown below does not need to import
anything. If you had python import sys by default and perhaps even create a
briefer alias for sys.stdin, then this gets shorter:
py -c "import sys; print(sum(int(F.split()[1])for F in sys.stdin))" On
Behalf Of Jon Ribbens via Python-list
Sent: Tuesday, November 7, 2023 11:06 AM
To: python-list@python.org
Subject: Re: Python Golf
On 2023-11-07, Stefan Ram wrote:
> I read this in a shell newsgroup:
>
> perl -anE '$s += $F[1]; END {say $s}' in
>
> , so I wrote
>
> py -c "import sys; print(sum(int(F.split()[1])for F in sys.stdin))"
> to show that this is possible with Python too.
>
> But now people complain that it's longer than the Perl version.
>
> Do you see ways to make it shorter (beyond removing one space
> after the semicolon ";")?
It's a bit of an unfair competition given that, unlike Perl,
Python is not designed to be an 'awk' replacement.
Having said that, you could make it a bit shorter:
py -c "print(sum(int(F.split()[1])for F in open(0)))" https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Just mildly noticing the topics discussed have wandered quite a bit away from Python, let alone even programming. Phone numbers are not what they used to be. They tend to be quite portable and in some ways can be chained so my house phone rings through to my cell phone but a text will not be forwarded. And, if I do not choose to read my texts, no amount of sending would enlighten me about your needs. I know people who get WhatsApp messages but not standard texts, for example. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Monday, November 6, 2023 6:20 PM To: python-list@python.org Subject: Re: Checking if email is valid On Tue, 7 Nov 2023 at 10:11, Greg Ewing via Python-list wrote: > > On 7/11/23 7:45 am, Mats Wichmann wrote: > > Continuing with the example, if you have a single phone number field, or > > let a mobile number be entered in a field marked for landline, you will > > probably assume you can text to that number. > > But if the site can detect that you've entered a mobile number into > the landline field or vice versa and reject it, then it can figure out > whether it can text to a given numner or not without you having > to tell it! > Oh yes, it totally can. Never mind that some mobile numbers are able to accept text messages but not calls, nor that some landline numbers can accept text messages as well as calls :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Grant (and others), I am asking about the overall programming process of dealing with email addresses beyond checking the string for some validity. You mentioned requiring you type in your email twice as one example. I generally do a copy/paste to avoid typing or have my browser fill it in. Rarely the code is set to force me to actually type it in. And I note sites that force me to do too much typing of any kind or make me jump through hoops like having to get an email or text with a secondary number to type in or make me look at pictures and find the right ones and so on, encourage me to not use them much. There is a price for taking away convenience even if you see it as some form of security. Yes, there are tradeoffs. It really may be important to know what you want from your email addresses. If I sign YOU up for something like the Word of the day in a dozen languages by supplying your valid email address, then checking if it looks valid is less useful than sending an email to that address and asking the recipient to opt-in and verify they legitimately want it. If you want to ensure that your newsletter is still wanted, you may do something similar every year or so to everyone, or perhaps just those that have not had activity. If a mailbox starts rejecting messages, perhaps you send messages to their secondary contact info or just remove them. There are many such strategies and some may be way harder to implement than a simple and perhaps simplistic syntax check. I do wonder how much it sometimes matters when we see real-world scenarios where people who died a decade ago remain on voter registration rolls. If my mailing list has a few hundred bad emails on it, the costs of sending may be small albeit dealing with rejection messages may clog my logs. As for fake email addresses, there are many ways to play that game that are unlikely to be caught. Will they realize there is nobody at erew...@gmail.com? If you want to know if someone is going to sell your hello.th...@gmail.com address could you supply hell.othe...@gmail.com and then monitor mail that you will still receive as it seems google ignores periods in your email name? And, since others generally see the above as distinct, you can even use such a method to sign up for something multiple times. Complexity leaves room for loopholes. Still, obviously there are good reasons to do what you can to do some validation at many points along the way and especially when it may be critical. Asking someone to type in a new password twice when they cannot easily see what they are typing, is obviously useful as the consequence of losing it is high. Are getting the email addresses right as important? I know my wife registered a fairly common name of the jane@gmail.com variety that is now useless as it keeps receiving messages someone provided or typed in wrong that were supposed to go to janedoe@ or doe.jane@ or janedoe123@ or j.doe@ and so on. These include receipts, subscriptions to newsletters and much more. Some are inadvertent but the reality is she stopped using that email as it is now mostly full of SPAM as the others ... -Original Message- From: Python-list On Behalf Of Grant Edwards via Python-list Sent: Sunday, November 5, 2023 12:39 AM To: python-list@python.org Subject: Re: Checking if email is valid On 2023-11-04, Michael Torrie via Python-list wrote: > On 11/4/23 02:51, Simon Connah via Python-list wrote: > >> Wow. I'm half tempted to make a weird email address to see how many >> websites get it wrong. In my experience, they don't have to be very weird at all. >> Thank you for the link. > > Nearly all websites seem to reject simple correct email addresses > such as myemail+sometext@example.domain. I like to use this kind of > email address when I can to help me filter out the inevitable spam > that comes from companies selling off my address even after claiming > they won't. I've always suspected that's intentional. They refuse those sorts of e-mail addresses because they know that's what they are used for. If they allowed "plus suffixed" e-mail addresses, then all the crap they want to send to you would go into /dev/null where it belongs -- and we can't have that! > So I suspect that nearly all websites are going to reject other > kinds of weird email addresses you can create that are actually > correct. Definitely. Syntactic e-mail address "validation" is one of the most useless and widely broken things on the Interwebs. People who do anything other than require an '@' (and optionally make you enter the same @-containing string twice) are deluding themselves. -- Grant -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Chris, I don't mean anything specific in the abstract approach I outlined as a possible alternative to using one complex regular expression. My suggestion was that some of the work could be done by the module you used and THEN you may test various parts of the rest. For example, perhaps another module lets you test if the domain name is registered. I have not read the RFC and have not worked on email applications in decades and am not offering specific advice. I am merely suggesting the possible use of existing software modules that may provide a better approach in weeding out SOME bad addresses. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Friday, November 3, 2023 1:43 AM To: python-list@python.org Subject: Re: Checking if email is valid On Fri, 3 Nov 2023 at 12:21, AVI GROSS via Python-list wrote: > My guess is that a first test of an email address might be to see if a decent module of that kind fills out the object to your satisfaction. You can then perhaps test parts of the object, rather than everything at once, to see if it is obviously invalid. As an example, what does u...@alpha...com with what seems to be lots of meaningless periods, get parsed into? > What do you mean by "obviously invalid"? Have you read the RFC? ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
I have never had a need to check email but in my reading over the years, I am aware of modules of multiple kinds you can use to do things like parsing dates, URL and email addresses and probably many other such things into some kind of object and then you can use aspects of the object to do interesting things and perhaps change some and then ask for the object to be placed back into some other format such as text. My guess is that a first test of an email address might be to see if a decent module of that kind fills out the object to your satisfaction. You can then perhaps test parts of the object, rather than everything at once, to see if it is obviously invalid. As an example, what does u...@alpha...com with what seems to be lots of meaningless periods, get parsed into? This may be another approach that reuses what may be well-designed and tested shared software. I wonder if there are also such modules that do quite a bit of what is asked which is to reject a large class of badly formed addresses. You could, of course, take what survives and run additional screens. In the end, this is a bit like junkmail where some light-AI algorithms go over a corpus of messages that humans have curated as junk or not junk and make some statistical decisions that are nonetheless often wrong. In that case, many humans nastily declare thinks as SPAM just because they do not want to get such messages. If you blasted out email alerts every time a child seems to have been kidnapped to everyone in the nation, how long before many such messages would become designated as SPAM? So is there any work where people have taken a decent collection of email addresses used in the past that turned out to be syntactically valid or not, and trained an algorithm to recognize most of them properly? That trained algorithm could be shared and incorporated into your programs either as the only method, or one you use in special cases. -Original Message- From: Python-list On Behalf Of Mike Dewhirst via Python-list Sent: Thursday, November 2, 2023 6:31 PM To: python-list@python.org Subject: Re: Checking if email is valid If i wanted an email verifier I would look at open source frameworks and see how they do it. Django comes to mind.--(Unsigned mail from my phone) Original message From: Michael Torrie via Python-list Date: 3/11/23 07:23 (GMT+10:00) To: python-list@python.org Subject: Re: Checking if email is valid On 11/2/23 00:42, Simon Connah via Python-list wrote:> Basically I'm writing unit tests and one of them passess in a string > with an invalid email address. I need to be able to check the string > to see if it is a valid email so that the unit test passess.If you truly have managed to code an RFC-compliant verifier, I commend you.> Valid as in conforms to the standard. Although having looked at the> standard that might be more difficult than originally planned.You'll have to read the relevant RFCs. Lots of corner cases! From whatI can see virtually no one on the internet gets it right, judging by thenumber of times I have valid email addresses flagged as not valid bypoor algorithms.-- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Yes, Chris, many things can be used for lesser purposes. Perhaps this could be like when people automate guessing passwords and one defense is to stop accepting after N bad guesses till some external method resets things. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Thursday, November 2, 2023 2:05 AM To: python-list@python.org Subject: Re: Checking if email is valid On Thu, 2 Nov 2023 at 15:20, AVI GROSS via Python-list wrote: > > Yes, it would be nice if there was a syntax for sending a test message sort > of like an ACK that is not delivered to the recipient but merely results in > some status being sent back such as DELIVERABLE or NO SUCH USER or even > MAILBOX FULL. > Yes, it would! Spammers would be able to use this syntax to figure out exactly which addresses actually have real people connected to it. It would save them so much trouble! Brilliant idea. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Checking if email is valid
Yes, it would be nice if there was a syntax for sending a test message sort
of like an ACK that is not delivered to the recipient but merely results in
some status being sent back such as DELIVERABLE or NO SUCH USER or even
MAILBOX FULL.
An issue with the discussion that may be worth considering is that some
email addresses are not always valid or may not be valid yet but will be
activated later. If I plan on opening a business unit which DNS will later
support as specific.category.mycompany.com.au and we first want to write
some code and test it and roll everything out later, then a test for
u...@specific.category.mycompany.com.au could fail some tests now but may be
fine later. Or what if I turn my machine off on weekends and when it boots,
it sets up to be able to receive mail. Is the address only sometimes valid?
We cannot be sure what rules may change and for all we know, they will
select other UNICODE symbols to replace @ for use by countries not having an
@ on keyboards in the local language or support some syntax like {AT} to be
usable ...
I even wonder about a service along the lines of tinyurl where you register
a potentially long or complex or hard to type name and get a short readable
one instead that is just used to provide a re-direct or even changed
periodically to dynamically point to where you want them now, such for the
current day of the week. I can easily imagine them making a funny looking
email address such as user@TINYqwerty that may not pas your current test or
one that looks valid to you but maps into an invalid or even null address.
BTW, checking if an email is valid is much wider as a concept than whether
the email address looks like a possible address. A big check sometimes made
if if the headers in the message and various formatting issues look
reasonable or issues about attachments and even if it is passed by SPAM
detectors. This discussion is just about if an email address LOOKS possibly
valid or should not be accepted.
I note earlier iterations of email had addressed like
mach1!mach2!mach3!ihnp4!mach5!mach6!user or even mach1!mach2!user@mach3 and
I remember tools that analyzed what other machines various machines claimed
to have a direct connection to and tried to figure out a connection from
your source to destination, perhaps a shorter one or maybe a less expensive
one. Hence machines like ihnp4 and various universities that were densely
connected to others got lots of traffic. In that scenario, validity had
another meaning.
-Original Message-
From: Python-list On
Behalf Of D'Arcy Cain via Python-list
Sent: Wednesday, November 1, 2023 9:57 PM
To: python-list@python.org
Subject: Re: Checking if email is valid
On 2023-11-01 17:17, Chris Angelico via Python-list wrote:
> On Thu, 2 Nov 2023 at 08:09, Grant Edwards via Python-list
> wrote:
>> Make sure it has an '@' in it. Possibly require at least one '.'
>> after the '@'.
>
> No guarantee that there'll be a dot after the at. (Technically there's
> no guarantee of an at sign either, but email addresses without at
> signs are local-only, so in many contexts, you can assume there needs
> to be an at.)
druid!darcy - doesn't work any more but not because it is syntactically
incorrect.
Remember the good old days when we were able to test if an address
existed without sending? That was before the black hats discovered the
Internet.
--
D'Arcy J.M. Cain
System Administrator, Vex.Net
http://www.Vex.Net/ IM:da...@vex.net
VoIP: sip:da...@vex.net
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
Thomas, It looks like much of our discussion and attempts at help are not going to be that helpful to Tenor as we may be way off bass about what he wants to do and certainly RSTUDIO and quite a few other suggestions may not be available in his microcontroller. As I see it, some of his objective involves sampling a sensor in real time. I have not heard what he wants to do with the data gathered and this may be an example of where code needs to be running fast enough to keep up. Proving the code will work, especially if you add logging or print statements or run it in a monitored mode so you can follow what it is doing, presents special challenges. Now if he ever wants to read in a .CSV file and analyze the data and make graphs and so on, I might chime in. For now, I am dropping out. Avi -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Thursday, October 26, 2023 6:50 PM To: python-list@python.org Subject: Re: Question(s) On 10/26/2023 6:36 PM, AVI GROSS via Python-list wrote: > I am not one for IDLE worship, Tenor. But if you have been getting a message here, it is that there are an amazing number of programs that support your use of python during the development phase and perhaps later. I actually often use an environment called RSTUDIO (now part of a new name of POSIT) because it has been expanded beyond supporting R and supports Python and a growing number of other languages or combos that combine word processing with inserts from multiple languages. Excellent! I didn't know about this development. [snip] -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
I am not one for IDLE worship, Tenor. But if you have been getting a message here, it is that there are an amazing number of programs that support your use of python during the development phase and perhaps later. I actually often use an environment called RSTUDIO (now part of a new name of POSIT) because it has been expanded beyond supporting R and supports Python and a growing number of other languages or combos that combine word processing with inserts from multiple languages. I have not used it for other languages like C/C++ and Javascript but the point is that like some others, it is not specific to Python but provides support for it. And, somewhat amusingly, you can write programs that combine parts in R and in Python that inter-operate with each other. Since you are taking a fairly overwhelming challenge of trying to learn everything at once while also developing something you want to be perfect and with few if any flaws, it may make sense to start with more of an ASCII editor or something with a few features but a simple interface, and a bit later when some parts of your code are working and you have some experience, you can move the code up a few notches to tools that perform a lot more validation for you. Please consider that resources trying to teach you the language, besides often showing simpler scenarios, often are not going to tell you EVERYTHING else you may need or that is usable let alone teach you all available modules and so on. -Original Message- From: Python-list On Behalf Of o1bigtenor via Python-list Sent: Thursday, October 26, 2023 8:34 AM To: Michael Torrie Cc: python-list@python.org Subject: Re: Question(s) On Wed, Oct 25, 2023 at 10:19 AM Michael Torrie via Python-list wrote: > > On 10/25/23 05:51, o1bigtenor via Python-list wrote: > > Looks like I have another area to investigate. (grin!) > > Any suggestions? > > Seems to me you're trying to run before you have learned to walk. > > Slow down, go to the beginning and just learn python, write some code, > see if it runs. Go through the tutorial at > https://docs.python.org/3/tutorial/index.html Interesting - - - - ". . . see if it runs." - - - that's the issue! When the code is accessing sensors there isn't an easy way to check that the code is working until one has done the all of the physical construction. If I'm trying to control a pulsation system using square waves with distinct needs for timing etc I hadn't seen any way of 'stepping through the code' (phrase you use later). > > Your first and most basic tool is the python interpreter. It will tell > you when you try to run your code if you have syntax errors. It's true > that some errors the linters will catch won't show up as syntax errors, > but cross the bridge when you get to it. Once you have a basic grasp of > Python syntax, you can begin using some of the tools Python has for > organizing code: Functions and modules (eventually packages). > Eventually when your logic is placed neatly into functions, you can then > write other python programs that import those functions and feed > different parameters to them and test that the output is what you > expect. That is known as a test. > > Nothing wrong with geany as an editor. However, you might find the > Python Idle IDE useful (it usually installs with Python), as it lets you > work more interactively with your code, inspecting and interacting with > live python objects in memory. It also integrates debugging > functionality to let you step through your code one line at a time and > watch variables and how they change. I have been following this list for some time. Don't believe that I've ever seen anything where anyone was referred to 'Idle'. In reading other user group threads I have heard lots about java and its ide - - - don't remember, again, any re: an ide for python. Even in maker threads - - - say for arduino - - its 'use this cut and paste method of programming' with no mention of any kind of ide when it was microPython - - although being a subset of python it Idle may not work with it. > > When you encounter isses with your code (syntax or logical) that you > can't solve, you can come to the list, show your code and the full > output of the interpreter that shows the complete error message and back > trace and I think you'll get a lot of helpful responses. > -- That was the plan. My problem is that I'm needing to move quite quickly from 'hello, world' to something quite a bit more complex. Most of the instruction stuff I've run into assumes that one is programming only for the joy of learning to program where I've got things I want to do and - - - sadly - - - they're not sorta like the run of the mill stuff. Oh well - - - I am working on things! Thanks for the ideas and the assistance! Regards -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
Just want to add that even when you can prove that an algorithm works absolutely positively, it will often fail on our every finite computers. Consider families of algorithms that do hill climbing to find a minimum and maximum and are guaranteed to get ever closer to a solution given infinite time. In real life, using something like 64 bit or 128 bit floating point representations, you can end up with all kinds of rounding errors and inexactness on some inputs so it goes into a loop of constantly bouncing back and forth between the two sides and never gets any closer to the peak. It may work 99.99% of the time and then mysteriously lock up on some new data. Or consider an algorithm some use in places like Vegas that is absolutely 100% guaranteed to win. If you bet $1 and lose, simply double your bet as many times as needed and eventually, you should win. Of course, one little problem is that all you ever win is $1. And you might lose 50 times in a row and spend hours and risk ever larger amounts to just win that dollar. Sure, you could just start betting a larger amount like a million dollars and eventually win a million dollars but how long can anyone keep doubling before they have to stop and lose it all. After enough doublings the vet is for billions of dollars and soon thereafter, more than the worth of everything on the planet. The algorithm is mathematically sound but the result given other realities is not. A last analogy is the division by zero issue. If your algorithm deals with infinitesimally smaller numbers, it may simply be rounded or truncated to exactly zero. The next time the algorithm does a division, you get a serious error. So perhaps a PROOF that a real computer program will work would require quite a few constraints. Python already by default supports integers limited only in size by available memory. This can avoid some of the overflow problems when all you are allowed is 64 bits but it remains a constraint and a danger as even a fairly simple algorithm you can PROVE will work, will still fail if your program uses these large integers in ways that make multiple such large integers on machines not designed to extend their memory into whole farms of machines or generates numbers like Googolplex factorial divided by googolplex raised to the log(Googolplex ) power. Some problems like the above are manageable as in setting limits and simply returning failure without crashing. Many well-designed programs can be trusted to work well as long as certain assumptions are honored. But often it simply is not true and things can change. Python actually is a good choice for quite a bit and often will not fail where some other environment might but there are few guarantees and thus people often program defensively even in places they expect no problems. As an example, I have written programs that ran for DAYS and generated many millions of files as they chugged along in a simulation and then mysteriously died. I did not bother trying to find out why one program it called failed that rarely to produce a result. I simply wrapped the section that called the occasional offender in the equivalent try/catch for that language and when it happened, did something appropriate and continued. The few occasional errors were a bug in someone else's code that should have handled whatever weird data I threw at it gracefully but didn't, so I added my overhead so I could catch it. The rare event did not matter much given the millions that gave me the analysis I wanted. But the point is even if my code had been certified as guaranteed to be bug free, any time I stepped outside by calling code from anyone else in a library, or an external program, it is no longer guaranteed. We are now spending quite a bit of time educating someone who seems to have taken on a task without really being generally knowledgeable about much outside the core language and how much of the answer to making the code as reliable as it can be may lie somewhat outside the program as just seen by the interpreter. And unless this is a one-shot deal, in the real world, programs keep getting modified and new features ofteh added and just fixing one bug can break other parts so you would need to verify things over and over and then freeze. -Original Message- From: Python-list On Behalf Of Michael F. Stemper via Python-list Sent: Wednesday, October 25, 2023 9:34 AM To: python-list@python.org Subject: Re: Question(s) On 24/10/2023 18.15, o1bigtenor wrote: > What is interesting about this is the absolute certainty that it is impossible > to program so that that program is provably correct. Not entirely true. If I was to write a program to calculate Fibonacci numbers, or echo back user input, that program could be proven correct. But, there is a huge set of programs for which it is not possible to prove correctness. In fact, there is a huge (countably infinite) set of programs
RE: Question(s)
I am replying to this which is heading into another topic: "(Tongue in cheek) Except doesn't one make more when software in hidden in an unreadable state? (That forces the user to go back to the original dev or group - - yes?)" Like some of us, I come from a time when much software was compiled. Sure, you could play interactively with the BASIC interpreter, but much else could be delivered to you as an executable and it was not easy to know what it did without trying it and certainly it was not easy to make changes to it and resell it as your own. Where does python fit in? On the one hand, it is completely visible as the software and modules you use tend to be stored on the machine being used in a readable state. If you come up with some nifty algorithm, it is there for anyone to see and copy or even alter if they have permissions. You can freely search a corpus of code to pick up interesting tidbits and that can be a plus but if your livelihood is based on selling your code or services, ... So do some people do things to make that harder? Can you deliver only files already converted to bytecode, for example? Could you have an interpreter that has special changes such as being able to take encrypted code and decrypt before using or perhaps have read privileges that normal users will not have? Obviously if your code is on a server that users can only access indirectly and in a controlled manner, this is not as much of an issue. I will skip the anecdotes, but point out how sometimes compiled code may have a whole bunch of other problems, including when a user can sneak in your office and modify the source code behind your back or when a virus can insert itself. So to return to the main point, not that I am selling anything, what do developers using Python do to try to make sure they get properly paid and others do not just use their work without permission? -Original Message- From: o1bigtenor Sent: Wednesday, October 25, 2023 6:59 AM To: avi.e.gr...@gmail.com Cc: Chris Angelico ; python-list@python.org Subject: Re: Question(s) On Tue, Oct 24, 2023 at 9:36 PM AVI GROSS via Python-list wrote: > > Agreed, Chris. There are many methods way better than the sort of RAID > architecture I supplied as AN EXAMPLE easy to understand. But even so, if a > hard disk or memory chip is fried or a nuclear bomb takes out all servers in > or near a city, you would need some truly crazy architectures with info not > only distributed across the globe but perhaps also to various space > satellites or servers kept ever further out and eventually in hyperspace or > within a black hole (might be write-only, alas). > > The point many of us keep saying is there can not easily or even with great > difficult, any perfect scheme that guarantees nothing will go wrong with the > software, hardware, the people using it and so on. And in the real world, as > compared to the reel world, many programs cannot remain static. Picture a > program that includes many tax laws and implementations that has to be > changed at least yearly as laws change. Some near-perfect code now has to > either be patched with lots of errors possible, or redesigned from scratch > and if it takes long enough, will come out after yet more changes and thus > be wrong. > > A decent question you can ask is if the language this forum is supposed to > be on, is better in some ways to provide the kind of Teflon-coated code he > wants. Are there features better avoided? How do you make sure updates to > modules you use and trust are managed as they may break your code. Stuff > like that is not as abstract. The above are very interesting questions - - - - anyone care to tackle one, or some? > > In my view, one consideration can be that when people can examine your > source code in the original language, that can open up ways others might > find ways to break it, more so than a compiled program that you only can > read in a more opaque way. > (Tongue in cheek) Except doesn't one make more when software in hidden in an unreadable state? (That forces the user to go back to the original dev or group - - yes?) TIA -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
Agreed, Chris. There are many methods way better than the sort of RAID architecture I supplied as AN EXAMPLE easy to understand. But even so, if a hard disk or memory chip is fried or a nuclear bomb takes out all servers in or near a city, you would need some truly crazy architectures with info not only distributed across the globe but perhaps also to various space satellites or servers kept ever further out and eventually in hyperspace or within a black hole (might be write-only, alas). The point many of us keep saying is there can not easily or even with great difficult, any perfect scheme that guarantees nothing will go wrong with the software, hardware, the people using it and so on. And in the real world, as compared to the reel world, many programs cannot remain static. Picture a program that includes many tax laws and implementations that has to be changed at least yearly as laws change. Some near-perfect code now has to either be patched with lots of errors possible, or redesigned from scratch and if it takes long enough, will come out after yet more changes and thus be wrong. A decent question you can ask is if the language this forum is supposed to be on, is better in some ways to provide the kind of Teflon-coated code he wants. Are there features better avoided? How do you make sure updates to modules you use and trust are managed as they may break your code. Stuff like that is not as abstract. In my view, one consideration can be that when people can examine your source code in the original language, that can open up ways others might find ways to break it, more so than a compiled program that you only can read in a more opaque way. -Original Message- From: Python-list On Behalf Of Chris Angelico via Python-list Sent: Tuesday, October 24, 2023 9:41 PM To: python-list@python.org Subject: Re: Question(s) On Wed, 25 Oct 2023 at 12:20, AVI GROSS via Python-list wrote: > Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error. > True, but there are FAR more efficient ways to do error correction :) Hamming codes give you single-bit correction and two-bit detection at a cost of log N bits, which is incredibly cheap - even if you were to go for a granularity of 64 bytes (one cache line in a modern Intel CPU), you would need just 11 bits of Hamming code for every 512 bits of data and you can guarantee to fix any single-bit error in any cache line. The "if most agree, ignore the one or two that don't" design implies that you're writing to an absolute minimum of three places, and in order to be able to ignore two that disagree, you'd probably need five copies of everything - that is to say, to store 512 bits of data, you would need 2560 bits of storage. But with a Hamming code, you need just 523 bits to store 512 reliably. Here's a great run-down on how efficiently this can be done, and how easily. https://www.youtube.com/watch?v=X8jsijhllIA Side note: If we assume that random bit flips occur at a rate of one every X storage bits, having redundant copies of data will increase the chances of a failure happening. For example, using a naive and horrendously wasteful "store 256 copies of everything" strategy, you would be 256 times more likely to have a random bitflip, which is insane :) You would also be able to guarantee detection of up to 128 random bitflips. But as you can see, this puts a maximum on your storage ratio. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Question(s)
Whoa! The question cannot be about whether it is possible to prove any abstract program will be correct and especially not on real hardware that can fail in various ways or have unexpected race conditions or interacts with other places such as over the internet. It has been quite well proven (think Kurt Gödel) that any system as complex as just arithmetic can have propositions that can neither be proven as true or as false and could be either. So there will be logical setups, written perhaps into the form of programs, that cannot be proven to work right, or even just halt someday when done. The real question is way more detailed and complex. How does one create a complex program while taking care to minimize as well as you can the chances it is flawed under some conditions. There are walls of books written on such topics and they range from ways to write the software, perhaps in small modules that can be tested and then combined into larger subunits that can also be tested. There are compilers/interpreters/linters and sometimes ways of declaring your intentions to them, that can catch some kinds of possible errors, or force you to find another way to do things. You can hire teams of people to create test cases and try them or automate them. You can fill the code with all kinds of tests and conditionals even at run time that guarantee to handle any kinds of data/arguments handed to it and do something valid or fail with stated reasons. You can generate all kinds of logs to help establish the right things are happening or catch some errors. But all that gets you typically is fewer bugs and software that is very expensive to create and decades to produce and by that time, you have lost your market to others who settle for less. Consider an example of bit rot. I mean what if your CPU or hard disk has a location where you can write a byte and read it back multiple times and sometimes get the wrong result. To be really cautions, you might need your software to write something in multiple locations and when it reads it back in, check all of them and if most agree, ignore the one or two that don't while blocking that memory area off and moving your data elsewhere. Or consider a memory leak that happens rarely but if a program runs for years or decades, may end up causing an unanticipated error. You can only do so much. So once you have some idea what language you want to use and what development environment and so on, research what tools and methods are available and see what you can afford to do. But if you have also not chosen your target architecture and are being asked to GUARANTEE things from afar, that opens a whole new set of issues. I was on a project once where we had a sort of networked system of machines exchanging things like email and we tested it. A while later, we decided to buy and add more machines of a new kind and had a heterogeneous network. Unfortunately, some tests had not been done with messages of a size that turned out to not be allowed on one set of machines as too big but were allowed on the other that had a higher limit. We caught the error in the field when a message of that size was sent and then got caught in junkmail later as the receiving or intermediate machine was not expecting to be the one dealing with it. We then lowered the maximum allowed size on all architectures to the capacity of the weakest one. This reminds me a bit of questions about languages that are free and come pretty much without guarantees or support. Is it safe to use them? I mean could they be harboring back doors or spying on you? Will you get a guarantee they won't switch to a version 3.0 that is incompatible with some features your software used? The short answer is there are no guarantees albeit maybe you can purchase some assurances and services from some third party who might be able to help you with the open-source software. Unless your project accepts the realities, why start? -Original Message- From: Python-list On Behalf Of o1bigtenor via Python-list Sent: Tuesday, October 24, 2023 7:15 PM To: Thomas Passin Cc: python-list@python.org Subject: Re: Question(s) On Tue, Oct 24, 2023 at 6:09 PM Thomas Passin via Python-list wrote: > snip > > By now you have read many responses that basically say that you cannot > prove that a given program has no errors, even apart from the hardware > question. Even if it could be done, the kind of specification that you > would need would in itself be difficult to create, read, and understand, > and would be subject to bugs itself. > > Something less ambitious than a full proof of correctness of an > arbitrary program can sometimes be achieved. The programming team for > the Apollo moon mission developed a system which, if you would write > your requirements in a certain way, could generate correct C code for them. > > You won't be doing that. > > Here I want to point out something else. You
RE: Where I do ask for a new feature
I still see no great reason for a new feature here and the namespace issue has often been discussed. You can always opt to create your own namespace of some sort and make many of your variables within it and always refer to the variables explicitly so the only collisions that can happen are your own carelessness. I do note that reusing a variable like "i" is not uncommon and especially when it is merely used as a looping variable. Generally anything else using the same variable does not need to refer to the other use and is in another scope. May I politely ask if you can point to other languages that have the feature you want and what it looks like. How does it know when the variable can safely go away? Does it allow you to create your alias in something like a loop and re-assign it or is it more like a constant once set and so on? If you have a good use case with no other easy way to provide it, you still need to show it is more important than oodles of other feature requests before anyone would consider seriously doing it in some future release. I am wondering if your concept of an alias is more typographical than actual. I mean in languages like C, there was often a preprocessor that went through your code and made changes like: #DEFINE filename "/usr/me/dir/subdir/file.c" This could allow some shorter typing but would not have anything in the namespace on the compiler level which would just see the longer substitutions. Could Python include something like this by keeping a table of symbols and replacements or running a pre-processor first? Maybe. But as stated, it does not seem to be a NEED that some feel is important. Assigning a variable to hold a pointer of sorts does indeed add to the namespace but the resource involved is not a big deal. Python is an interpreted language that makes one pass but there are languages that make multiple passes through the code and allow things like defining a function after it has been called. That is not pythonic. -Original Message- From: Python-list On Behalf Of Roel Schroeven via Python-list Sent: Friday, October 20, 2023 3:55 AM To: python-list@python.org Subject: Re: Where I do ask for a new feature Op 20/10/2023 om 5:16 schreef Bongo Ferno via Python-list: > On Thursday, October 19, 2023 at 11:26:52 PM UTC-3, avi.e...@gmail.com wrote: > > > There are many ways to make transient variables that disappear at some time > > and do we need yet another? Yes, you can create one of those ways but what > > is the big deal with deleting a variable when no longer used? > > Assigning a variable to something can be anything else than a temporal alias. > A with statement makes clear that the alias is an alias and is local, and it > automatically clears the variable after the block code is used. > > Python clutters the variable space with vars that are needed only on certain > places, and an alias doesn't has a scope. > Convenient alias are short names, and short names are limited in quantity. If > the space is cluttered with short alias, it opens risks for wrong utilization. > > Its like writing a "for i" in a list comprehension and having to worry if "i" > was already used in another place.. As long as functions are kept reasonably short, which is a good idea anyway, I don't really see any of that as a problem. -- "Experience is that marvelous thing that enables you to recognize a mistake when you make it again." -- Franklin P. Jones -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Where I do ask for a new feature
Bongo, Variables in most programming languages either have to be removed manually or allowed to drift outside a boundary when they disappear for scoping reasons and perhaps are garbage collected at some point. There are many ways to make transient variables that disappear at some time and do we need yet another? Yes, you can create one of those ways but what is the big deal with deleting a variable when no longer used? Examples might be the "finally" clause or the "with" statement or just putting the variable in a nested scope. -Original Message- From: Python-list On Behalf Of Bongo Ferno via Python-list Sent: Thursday, October 19, 2023 9:33 PM To: python-list@python.org Subject: Re: Where I do ask for a new feature > You can actually just do that with simple assignment! > > short_view = my_object.stuff.long_stuff.sub_object > print(short_view.some_method()) but then have to delete the variable manually del short_view -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Postgresql equivalent of Python's timeit?
Timing things that are fairly simple is hard enough to do repeatedly, but when it involves access to slower media and especially to network connections to servers, the number of things that can change are enormous. There are all kinds of caching at various levels depending on your hardware and resource contention with other programs running here and there as well as on various network-like structures and busses or just hard disks. Asking for anything to be repeated multiple times in a row as a general rule can make your results seem slower or faster depending on too many factors including what else is running on your machine. I am wondering if an approach to running something N times that may average things out a bit is to simply put in a pause. Have your program wait a few minutes between attempts and perhaps even do other things within your loop that make it likely some of the resources you want not to be in a queue have a chance to be flushed as other things take their place. Obviously, a machine or system with lots of resources may take more effort to use enough new data that replaces the old. Good luck. Getting reliable numbers is no easy feat as someone else may have trouble duplicating the results with a somewhat different setup. -Original Message- From: Python-list On Behalf Of Albert-Jan Roskam via Python-list Sent: Sunday, September 17, 2023 5:02 AM To: Peter J. Holzer Cc: python-list@python.org Subject: Re: Postgresql equivalent of Python's timeit? On Sep 15, 2023 19:45, "Peter J. Holzer via Python-list" wrote: On 2023-09-15 17:42:06 +0200, Albert-Jan Roskam via Python-list wrote: >This is more related to Postgresql than to Python, I hope this is ok. >I want to measure Postgres queries N times, much like Python timeit >(https://docs.python.org/3/library/timeit.html). I know about EXPLAIN >ANALYZE and psql \timing, but there's quite a bit of variation in the >times. Is there a timeit-like function in Postgresql? Why not simply call it n times from Python? (But be aware that calling the same query n times in a row is likely to be unrealistically fast because most of the data will already be in memory.) = Thanks, I'll give this a shot. Hopefully the caching is not an issue if I don't re-use the same database connection. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Fwd: AUTO EDITOR DIDN'T WORK
I think it is time to ask this topic to go find some other place to talk to itself. I have seen NO reason to think any question about problems with Python has been asked. Not properly. It sounds like someone messed up an installation, perhaps of other programs like an editor and some unspecified thing does not work. I suggest they start over and be careful so that if some specific version of Python is needed, it gets installed in the right place and so on. We here cannot be expected to have much idea about programs that perhaps we do not use. Python can be used to build an editor, or parts it runs when needed, and it can be used to create or consume XML, or do things with audio formats. So can many other languages. If it was needed here and the setup was wrong or sabotaged, that is not something easily handled here. Or did I miss something? If so, I know others here also missed it too. If there is a more specific problem like some lines of actual python code not doing what was expected, please share that specifically with enough detail. -Original Message- From: Python-list On Behalf Of Thomas Passin via Python-list Sent: Tuesday, June 13, 2023 10:18 PM To: python-list@python.org Subject: Re: Fwd: AUTO EDITOR DIDN'T WORK On 6/13/2023 9:43 PM, gene heskett via Python-list wrote: > On 6/13/23 19:10, Thomas Passin via Python-list wrote: >> On 6/13/2023 5:32 PM, Alan Gauld via Python-list wrote: >>> Okay thanks. Meanwhile, I am not tech savvy so I may not say much here. >>> I followed all the commands as given on the website to install auto >>> editor standing it on python but after rendering the XML file, I >>> couldn't open it with my Davinci Resolve 18. I uninstalled and >>> reinstalled about twice and still no success hence I uninstalled it. >> >> I don't understand when you talk about an "XML file". Auto-editor >> works on video files, or at least .mp4 files, which are not XML files. >> Davinci Resolve does have some ability to interoperate with other >> editors using XML in some way (according to Wikipedia, >> https://en.wikipedia.org/wiki/DaVinci_Resolve) but that's a different >> thing completely. >> >> I also don't know what you mean by "after rendering the XML file" >> since from what I can see auto-edit doesn't render anything. >> >> The simplest thing that auto-editor can do is to cut out long periods >> of dead space, e.g., from an mp4 file. Their documentation shows how >> to do it. If it were me, I would run the example command line on a >> sample mp4 file, then see what it looked like in Davinci. Is that >> what you did? It should be the same video but with some dead space >> removed. >> >> (Note that I'm speaking from a place of no experience with either of >> these software packages; just looking at what auto-edit claims to do). >> >> >> > auto-edit? Never heard of it. xml? I've written hundred of kilobytes of > it in plain old geany. I didn't know there was a special editor for xml. Oh, there are, there are - mostly intended for document authoring, I think. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Passing information between modules
There is no guarantee that argv is consulted earlier in the program than other modules will use it for communication. Consider a case where a program does look at argv but later wants to call another program using some or all of the components of argv and now there are added components there. That could lead to all kinds of problems. Some languages have global objects available that you can add to, sort of like a dictionary object and as long as you add keys guaranteed to be unique, can be used carefully to coordinate between parts of your program. Of course, you may also need to use techniques that ensure atomic concurrency. Reusing argv is a hack that should not be needed. On Sat, Nov 19, 2022, 4:37 PM Thomas Passin wrote: > On 11/19/2022 4:28 PM, Thomas Passin wrote: > > On 11/19/2022 3:46 PM, Michael F. Stemper wrote: > >> On 18/11/2022 04.53, Stefan Ram wrote: > >>>Can I use "sys.argv" to pass information between modules > >>>as follows? > >>> > >>>in module A: > >>> > >>> import sys > >>> sys.argv.append( "Hi there!" ) > >>> > >>>in module B: > >>> > >>> import sys > >>> message = sys.argv[ -1 ] > >> > >> I just tried and it appears that one can append to sys.argv. However, > >> it seems like an incredibly bad idea. > > > > For that matter, you can just directly add attributes to the sys module, > > no need to use sys.argv: > > > > >>> import sys > > >>> sys._extra = 'spam' # Not an exception > > >>> print(sys._extra) > > spam > > > > Probably not the best idea, though. Better to use some module that you > > control directly. > > This could be one of those things of which Raymond Chen (The Old New > Thing) asks "what if everyone did this?". Imagine if every > (non-standard-library) module misused sys or sys.argv like this. The > result could be chaotic. > > Best to put all your own stuff into modules that you yourself control. > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
Interesting idea, Anton. I would be interested in hearing more detail on how it would work. Although much of programming has been centered on the Latin alphabet and especially English, that may change. I can imagine a customized compiler or interpreter that uses key words in the local language instead of for or while or if or else or even import. If a region of UNICODE was set aside, would it have to be as a sort of additional ALT or shift key for anything, or just English characters or would it be for abstract symbols that would be mapped to and from a long list of reserved key words that may vary by locale? A serious problem I have been noting is the interactions between typographical paradigms never expected to interact. I mean special characters allowed or expected that clash. An example in Python and many other languages is regular expressions. When using a string of characters, you may have to deal with the rules for evaluating a string of that type and double up backslash characters or be carefully as a dollar sign or curly brace might invoke a side effect and interpolate something into it. But for languages that allow RE to be entered without a string as in /.../ you have to write it differently. I noted other ways in which naming conventions and even sort of keywords need special care. Consider a Javascript program in a browser that can read and manipulate HTML and CSS. The rules for names in JS do not allow hyphens while others often do. So the programmers making the structures in the DOM had to twist things. To access something like font-type you ask for font Type as an example by changing the identifier to camel case. I sometimes have programs that combine R and Python somewhat interchangeably and it highlights places the naming does not match. R allows periods as parts of names as in my.data and almost anything including spaces if you use grave accent quotes such as `one [identifier` so it is perfectly valid to have a function call like `[`(x, 5) to mean x[5] albeit explaining why that is useful is beyond the scope. Similarly you can make all kinds of in-line functions between percent signs as in %*% or %specialized computation% and you sort of make your own keywords albeit Chris and others may rightly suggest they are something else rather than the first level of syntax. My point is that your idea may need to support keywords for disparate ideas and languages. On Sun, Oct 16, 2022, 6:20 AM Antoon Pardon wrote: > Op 16/10/2022 om 00:50 schreef avi.e.gr...@gmail.com: > > This has been discussed so often precisely because I swear NO CHOICE of > keyword would satisfy everybody! Most languages start with designated > keywords and some reserve a few for later use. But then things can get > frozen in place to avoid breaking existing programs or break older > compilers/interpreters. > > > > Some languages use techniques to extend themselves more harmlessly such > as creating a singleton object that has content that can be regular data as > in math.pi, or functions/methods or new ides like "Symbols" that allow all > kinds of extensions to the language in a fairly harmless way as no older > program would likely have used features that did not exist. > > > > That might not easily solve this problem. But I wonder if reserving some > kind of prefix might help, so anything like extension.0nNoBreak could be > added to a loop as a final clause and be treated as a non-key keyword of > sorts. > > My idea would be to reserve different unicode blocks for the keywords > and the identifiers. e.g. We could reserve the mathematical alphanumeric > block for keywords and all other letters and numbers for identifiers. > Doing so would allow extenting the keywords without breaking programs > that already use that combination as an identifier. > > Python could slowly transition in this direction by first allowing the > current keywords to be in this block. Every new keyword would only be in > that unicode block. If would then be possible to write python code with > this convention but it wouldn't be obligatory. After some time the > python developers could decide to make it obligatory. > > I doubt this will idea will get from the ground, but I think it would > allow for a smoother transition into new concepts, as it is no longer a > strugle searching for a keyword that will break as little programs as > possible. > > -- > Antoon Pardon. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
I will say that those of us meaning me, who express reservations are not arguing it is a bad idea to get more info in one sweep. Many errors come in bunches. If I keep calling some function with the wrong number or type of arguments, it may be the same in a dozen places in my code. The first error report may make me search for the others places so I fix it all at once. Telling me where some instances are might speed that a bit. As long as it is understood that further errors are a heuristic and possibly misleading, fine. But an error like setting the size of a fixed length data structure to the right size may result in oodles of errors about being out of range that magically get fixed by one change. Sometimes too much info just gives you a headache. But a tool like you described could have uses even if imperfect. If you are teaching a course and students submit programs, could you grade the one with a single error higher than one with 5 errors shown imperfectly and fail the one with 600? On Sun, Oct 9, 2022, 1:53 PM Antoon Pardon wrote: > > > Op 9/10/2022 om 19:23 schreef Karsten Hilbert: > > Am Sun, Oct 09, 2022 at 06:59:36PM +0200 schrieb Antoon Pardon: > > > >> Op 9/10/2022 om 17:49 schreef Avi Gross: > >>> My guess is that finding 100 errors might turn out to be misleading. > If you > >>> fix just the first, many others would go away. > >> At this moment I would prefer a tool that reported 100 errors, which > would > >> allow me to easily correct 10 real errors, over the python strategy > which quits > >> after having found one syntax error. > > But the point is: you can't (there is no way to) be sure the > > 9+ errors really are errors. > > > > Unless you further constrict what sorts of errors you are > > looking for and what margin of error or leeway for false > > positives you want to allow. > > Look when I was at the university we had to program in Pascal and > the compilor we used continued parsing until the end. Sure there > were times that after a number of reported errors the number of > false positives became so high it was useless trying to find the > remaining true ones, but it still was more efficient to correct the > obvious ones, than to only correct the first one. > > I don't need to be sure. Even the occasional wrong correction > is probably still more efficient than quiting after the first > syntax error. > > -- > Antoon. > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
Peter, There can be excellent reasons to undo a pipeline like I described. I often write it carefully in smaller chunks while debugging and make it more elegant later ... But someone amused me by explaining they were going to let people believe the code was written by them so it had to fit their style and abilities. That meant removing most of my comments, renaming some variables, taking out code that checked things like whether a file existed before opening it and of course no pipelines. It had to be downgraded and had I known, I could have easily given them code written as if it was in some poorer language. Python has idioms often used in making pipes of a sort but in languages with other forms, such as R, debugging is not that difficult as you can insert functions in middle of a pipeline that print what you want but return the data structure they were fed for the next step in the pipeline. When done, remove the lines with such entries or change the function definition or something like that. Objects used as pipelines do not do this as easily as you may need to add methods ... On Sun, Oct 9, 2022, 1:17 PM Peter J. Holzer wrote: > On 2022-10-09 12:34:22 -0400, Avi Gross wrote: > > I have seen programmers who have taken an elegant pipeline I have built > > apart and made it into many lines of code reassignment the value of each > > step to the same or different variables and other ways of lengthening or > > obscuring my intent. > > I have certainly done that (not with your code, AFAIK). The problem with > those beautiful one-liners is that they are really hard to debug. So if > I can't convince myself that they are correct just by reading them I > have to split them over multiple lines so I can add breakpoints or log > messages. Of course I could put it together again afterwards, but I > would argue that if I didn't understand it the first time it's probably > better to leave it in its more verbose and debuggable state. > > Of course I have also done the opposite: Taken some messy and > complicated code and simplified it into a simple generator expression. > In fact I would say that I code tends to be shorter after I fixed a bug > than before. > > > > So although size may matter, so can sighs. > > :-) > > hp > > -- >_ | Peter J. Holzer| Story must make more sense than reality. > |_|_) || > | | | h...@hjp.at |-- Charles Stross, "Creative writing > __/ | http://www.hjp.at/ | challenge!" > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
Antoon, it may also relate to an interpreter versus compiler issue. Something like a compiler for C does not do anything except write code in an assembly language. It can choose to keep going after an error and start looking some more from a less stable place. Interpreters for Python have to catch interrupts as they go and often run code in small batches. Continuing to evaluate after an error could cause weird effects. So what you want is closer to a lint program that does not run code at all, or merely writes pseudocode to a file to be run faster later. Many languages now have blocks of code that are not really be evaluated till later. Some code is built on the fly. And some errors are not errors at first. Many languages let you not declare a variable before using it or allow it to change types. In some, the text is lazily evaluated as late as possible. I will say that often enough a program could report more possible errors. Putting your code into multiple files and modules may mean you could cleanly evaluate the code and return multiple errors from many modules as long as they are distinct. Finding all errors is not possible if recovery from one is not guaranteed. Take a language that uses a semicolon to end a statement. If absent usually there would be some error but often something on the next line. Your evaluator could do an experiment and add a semicolon and try again. This might work 90% of the time but sometimes the error was not ending the line with a backslash to make it continue properly, or an indentation issue and even spelling error. No guarantees. Is it that onerous to fix one thing and run it again? It was once when you handed in punch cards and waited a day or on very busy machines. On Sun, Oct 9, 2022, 1:03 PM Antoon Pardon wrote: > > > Op 9/10/2022 om 17:49 schreef Avi Gross: > > My guess is that finding 100 errors might turn out to be misleading. If > you > > fix just the first, many others would go away. > > At this moment I would prefer a tool that reported 100 errors, which would > allow me to easily correct 10 real errors, over the python strategy which > quits > after having found one syntax error. > > -- > Antoon. > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
Fair enough, Chris. There may be some overlap with the size of code for the most common cases but sometimes the opposite as those may be more complex to deal with. A reality for many programmers today is to not micromanage too early as things are often fast enough and any tweaking is best done only in critical areas. The emphasis may be on the programmer experience in writing fast code with fewer errors. Perhaps secondary but often important is making the code maintainable and in my experience that can often be best done by choosing meaningful names and brief selective comments than by worrying about the size of blocks of code. But others obviously preach what they think works for them even when it may constrain others more than it helps. I have seen people suggest that all variables have short names like a3 but that does not mean it improves anything other than the size of the code and parsing it. The loss in readability and so on probably is worse. On Sun, Oct 9, 2022, 12:53 PM Chris Angelico wrote: > On Mon, 10 Oct 2022 at 03:46, Avi Gross wrote: > > > > Chris, I was not arguing that at all. > > Maybe not intentionally, but you did lend a lot of weight to that argument > :) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
Chris, I was not arguing that at all. I was saying some rationales about how to order choices exist based on ideas like efficiency or other considerations. Sometimes people are mistaken as something may take constant time as implemented. And yes, many rules have countless exceptions. For example, if something is expected to rarely or never happen, code within that branch may not be needed to be optimized in any way as long as it works in the remote chance it is called. I think what was suggested here is more about code readability considerations and for some of us, making us stand on our heads to puzzle things out is harder than ordering longer items ... On Sun, Oct 9, 2022, 12:30 PM Chris Angelico wrote: > On Mon, 10 Oct 2022 at 03:22, Avi Gross wrote: > > > > Smallest code blocks first may be a more modern invention. > > > > Some would argue for a rule related to efficiency of execution. When you > > have multiple blocks as in an if-else or case statement with multiple > > choices, that you order the most common cases first. Those shorten > > execution more often than the rarer cases especially the ones that should > > never happen. > > > > Seems fairly dubious and full of special-cases. If you want to follow > that rule, it should be easy enough to still permit for-else clauses. > It's an extremely weak argument against for-else. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
Since many languages allow placing multiple statements on one line or spreading one over many lines, it seems that the number of lines in code can be adjusted. If I have a line like: Alpha, beta, gamma, delta = 1, 2, 3, 4 Could that be rewritten as 4 or more lines? I have seen programmers who have taken an elegant pipeline I have built apart and made it into many lines of code reassignment the value of each step to the same or different variables and other ways of lengthening or obscuring my intent. So although size may matter, so can sighs. On Sun, Oct 9, 2022, 4:24 AM Peter J. Holzer wrote: > On 2022-10-09 05:37:59 +0100, Axy via Python-list wrote: > > Actually the reason I never used "else" was the violation of the rule > > of beauty "shortest block first". > > That's a weird rule. > > I can see justifications for "most common case first" and "most special > case first", but ordering the cases in an if/elif/else statement by > length seems like ordering books by color: It may be pretty, but it > doesn't make them easy to find. > > hp > > -- >_ | Peter J. Holzer| Story must make more sense than reality. > |_|_) || > | | | h...@hjp.at |-- Charles Stross, "Creative writing > __/ | http://www.hjp.at/ | challenge!" > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
Smallest code blocks first may be a more modern invention. Some would argue for a rule related to efficiency of execution. When you have multiple blocks as in an if-else or case statement with multiple choices, that you order the most common cases first. Those shorten execution more often than the rarer cases especially the ones that should never happen. There are obvious exceptions like the default having to be last, albeit some languages allow the default to be inserted anywhere visually even if the code sort of runs last. But negating a condition so smaller code appears first may have some cost. I mean if !function() may be slower as the negating is an extra step. But it may be even slower if the inversion is done using a wrapper function that simply inverts the return value from the other function. I think sometimes a comment placed carefully that explains the code and logic in concise form is a simpler approach that can be followed by a big chunk then little chunk without loss of readability. In the original example the else part can be mentioned before the loop as a sort of reminder. In my experience, the size of code often varies within a project so a smaller chunk may grow as requirements change, such as adding debug or logging, and large chunks can shrink as common parts of the code get extracted into functions. So not a rule but realistically not always a bad idea to write code in a way that draws the attention of readers along the main path of execution and perhaps not showing all the checking for odd cases first. I mean as an example if the argument is of type text then do stuff, else if a number else if a symbol else if empty ... On Sun, Oct 9, 2022, 1:18 AM Chris Angelico wrote: > On Sun, 9 Oct 2022 at 16:05, Axy via Python-list > wrote: > > > > > > On 09/10/2022 05:47, Chris Angelico wrote: > > > On Sun, 9 Oct 2022 at 15:39, Axy via Python-list < > python-list@python.org> wrote: > > >> Got it, thanks! > > >> > > >> Actually the reason I never used "else" was the violation of the rule > of > > >> beauty "shortest block first". With if--else you can easily follow > this > > >> rule by inverting "if" expression, but with for--else you can't. The > > >> loop body of the simplest example is already three lines, in real life > > >> things are much worse. > > >> > > > That's not a rule I've ever been taught; how important is it? > > > > > > ChrisA > > > > It gets important if the lifetime of your project is more than three > > months and is extremely important if more than 10 years. But, it depends. > > Yes, I'm aware that code readability becomes irrelevant for > short-duration projects. Beside the point. I'm wondering how important > it really is to have the shortest block first. > > > I also might be wrong in terminology, anyway, there are many rules that > > make programmer's life easier, described in the literature from the old > > good "How to write unmaintainable code" to "The Art of Readable Code". > > And I hope there are a lot of recent books on this subject I did not > > track and read yet. > > Also not really a justification for "shortest block first". Wanting > some elaboration on that. What's the value in it? > > Given that for-else is an excellent, if rarely-used, construct, I > would say that, *at least*, it is worth setting aside this rule for > that particular situation. It is also generally worth using fewer > commas than I just did. Take my advice with a grain of salt. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
Anton There likely are such programs out there but are there universal agreements on how to figure out when a new safe zone of code starts where error testing can begin? For example a file full of function definitions might find an error in function 1 and try to find the end of that function and resume checking the next function. But what if a function defines local functions within it? What if the mistake in one line of code could still allow checking the next line rather than skipping it all? My guess is that finding 100 errors might turn out to be misleading. If you fix just the first, many others would go away. If you spell a variable name wrong when declaring it, a dozen uses of the right name may cause errors. Should you fix the first or change all later ones? On Sun, Oct 9, 2022, 6:11 AM Antoon Pardon wrote: > I would like a tool that tries to find as many syntax errors as possible > in a python file. I know there is the risk of false positives when a > tool tries to recover from a syntax error and proceeds but I would > prefer that over the current python strategy of quiting after the first > syntax error. I just want a tool for syntax errors. No style > enforcements. Any recommandations? -- Antoon Pardon > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Local variable definition in Python list comprehension
Dumb question. Your y is purely a function of x. So create an f(x) where you want your y. It probably can even be anonymous inline. I mean your return values of (x, y) would be (x, f(x)) ... On Thu, Sep 1, 2022, 5:04 PM Chris Angelico wrote: > On Fri, 2 Sept 2022 at 06:55, James Tsai wrote: > > > > 在 2022年9月1日星期四 UTC+2 18:34:36， 写道： > > > On 9/1/22, James Tsai wrote: > > > > > > > > I find it very useful if I am allowed to define new local variables > in a > > > > list comprehension. For example, I wish to have something like > > > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > > > > > For now this functionality can be achieved by writing > > > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > > > You can assign a local variable in the `if` expression. For example: > > > > > > >>> [(x, y) for x in range(10) if x + (y := x**2) < 30] > > > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)] > > > > Yeah this works great but like [(x, y) for x in range(10) for y in > [x**2]] I written before, is kind of a hack. And if initially I do not need > an "if" condition in the list comprehension, this becomes less convenient. > I still can write > > >>> [(x, y) for x in range(10) if (y := x**2) or True] > > > > But I wonder if Python could have a specific syntax to support this. > > > > But why would you need to assign to y in that example? If you're using > it more than once, you can use :=, and if you aren't, you don't need > to. But do be aware that := does not create a comprehension-local name > binding, but a nonlocal instead. > > > No but very often when I have written a neat list/dict/set > comprehension, I find it very necessary > > to define local variable(s) to make it more clear and concise. Otherwise > I have to break it down > > to several incrementally indented lines of for loops, if statements, and > variable assignments, > > which I think look less nice. > > Well, if it's outgrown a list comp, write it on multiple lines. Like I > said, not everything has to be a one-liner. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: what's the problem??????
Nati,
This is a two-way process and requires you to be very clear on what is not
working or what you are trying to do or help clear away having us try to
understand lots of code that is not very related to the question.
Your code, as shown, makes an empty string repeatedly in a loop.
a=dict()
I am guessing the code there works fine and does nothing useful. Well, what do
you want in your dictionary? Most people create a dictionary outside the loop
as an empty dictionary and then in the loop use one of many methods to add
key:value pairs.
Earlier in the code you had another line:
a=labels.to_dict()
If the labels variable had a method like that, that is also a way.
So be specific about what LINE or region of code and what is wrong and what you
already tried or error messages you got.
Avi
-Original Message-
From: נתי שטרן
To: Neuroimaging analysis in Python ;
python-list@python.org
Sent: Wed, Jul 13, 2022 2:35 pm
Subject: Re: what's the problem??
I want to set dict
בתאריך יום ד׳, 13 ביולי 2022, 20:47, מאת נתי שטרן :
> CODE:
>
> for nii in os.listdir("c:/users/administrator/desktop/nii"):
>
> from nilearn import plotting
> from nilearn import datasets
> atlas = datasets.fetch_atlas_msdl()
> # Loading atlas image stored in 'maps'
> atlas_filename =
> "C:/Users/Administrator/Desktop/64/64/2mm/maps.nii.gz"
> # Loading atlas data stored in 'labels'
> labels = pd.read_csv(
> "C:/Users/Administrator/Desktop/64/64/labels_64_dictionary.csv")
> a=labels.to_dict()
> b=a["Difumo_names"]
> from nilearn.maskers import NiftiMapsMasker
> masker = NiftiMapsMasker(maps_img=atlas_filename, standardize=True,
> memory='nilearn_cache', verbose=5)
>
> time_series = masker.fit_transform(
> "c:/users/administrator/desktop/nii/"+nii)
> try:
> from sklearn.covariance import GraphicalLassoCV
> except ImportError:
> # for Scitkit-Learn < v0.20.0
> from sklearn.covariance import GraphLassoCV as GraphicalLassoCV
>
> estimator = GraphicalLassoCV()
> estimator.fit(time_series)
> # Display the covariancec
> aas={}
> jsa=0
> for i in estimator.covariance_:
> r=list(a["Difumo_names"].values())[jsa]
> jsa=jsa+1
> a=dict()
>
>
> for x in range(64):
> g=list(a["Difumo_names"].values())[x]
>
> print(aas)
> t= nilearn.plotting.plot_img(estimator.covariance_, labels=list(a[
> "Difumo_names"].values()),
> figure=(9, 7), vmax=1, vmin=-1,
> title='Covariance')# The covariance can be found
> at estimator.covariance_
>
> # The covariance can be found at estimator.covariance_
> t2= nilearn.plotting.plot_matrix(estimator.covariance_, labels=list(a
> ["Difumo_names"].values()),
> figure=(9, 7), vmax=1, vmin=-1,
> title='Covariance')
>
>
>
> --
>
>
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: NILEARN - WHY THIS CODE THROWS AN ERROR?????
Nati Stern has asked several questions here, often about relatively technical
uses of python code that many of us have never used and still is not providing
more exact info that tends to be needed before anyone can even think of
diagnosing the problem.
I have learned to stay away from some such questioners. But I am wondering if
some people (others too) think this forum is a generalized help desk staffed by
College Professors with nothing else to do.
Many questions are best handled locally where people can look over your
shoulder or use the same software and may have some fluency in your native
language. And sometimes you need to do more investigating on your own, and
perhaps tell us what you tried and why it was not useful, or we end up making
endless suggestions and being told we are not working on your real issue and so
on.
The code below is just babel or maybe babble. Something nested in a loop had a
problem. Why not try something drastic and look at the files and PICK ONE and
use it step by step and see when it fails?
It looks like the code wants to ask for all files then ignore some.
Why you would import numpy repeatedly in a loop is beyond me! LOL!
But which command line failed? My GUESS is:
data = img.get_fdata()
If so, did you try to see the current value of the filename you call "i" in the
loop and see what name was loaded in what looks like a file ending in .nii in
this code:
img = nib.load(path+"/"+i)
You need to proceed step by step and see if any previous steps failed.
But what is possible is you got a file with .nii in middle of the name that
does not end in .gz, or is not in the format needed.
Good luck,
אבי גרוס
-Original Message-
From: MRAB
To: python-list@python.org
Sent: Fri, Jul 8, 2022 4:47 pm
Subject: Re: NILEARN - WHY THIS CODE THROWS AN ERROR?
On 08/07/2022 14:06, נתי שטרן wrote:
> fullcode:
>
>
>
> import nilearn.plotting as plot
> import os,gzip,io
> import nibabel as nib
> path="C:/users/administrator/desktop/nii"
> path2="C:/users/administrator/desktop/nii/out/"
> for i in os.listdir(path):
> if(".nii.gz" in i):
> pass
> else:
>
> if(".nii" in i):
> img = nib.load(path+"/"+i)
> data = img.get_fdata()
> print(data)
> import imageio
> X=0
> for s in data:
> import numpy
> aleph=numpy.array(s,dtype=numpy.int8)
> X=X+1
> plot.plot_img(aleph)
>
> imageio.imwrite("C:\\users\\administrator\\desktop\\nii\\"+i
> +str(X)+'.jpg', s)
>
>
>
>
>
>
> error:
> Data given cannot be loaded because it is not compatible with nibabel
> format
>
What's the complete traceback?
It might help you to identify the problem if you add messages to tell
you what it's doing, e.g. what file it's about to load.
Apparently, one of the files is not compatible with nibabel.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
David, I understand now. As a project for your own edification I can understand it, albeit it is a more substantial effort than many people might choose, LOL! So unless it starts being used heavily and adopted by some organization, the result of your effort will not necessarily be compatible with many modules now available or keep up with changes as python adds features or fixes bugs. I am curious about why something like numpy could not be integrated into what you do. Of course, if you are the only user, ... My hobbies to spend my time may not be as ambitious, but are quite a bit more varied! LOL! -Original Message- From: David J W To: Avi Gross Cc: python-list@python.org Sent: Fri, Jun 24, 2022 11:57 am Subject: Re: "CPython" The main motivation for a Python virtual machine in Rust is to strengthen my knowledge with Rust which currently has some gnarly bits to it but otherwise is an impressive low level language. Rust's future is looking very bright as even Linus Torvalds agrees with most of its design choices and is allowing it to be used as a linux kernel module language. Skipping ahead to the subject of names, Rython was chosen because "Python" is trademarked by the PSF so anything with the complete word Python in it is out. A close runner up would have been Camelot but that is already taken. Going backward to the issue of use and audience. Making Rython a real virtual machine that passes the CPython unit-tests is the only goal. I am actively following the faster CPython fork that Mike Shannon, GVR, and others are working on with the intention to try and incorporate what they discover into my project but I don't think Rython will be dramatically faster than Cpython because I am going to implement the same PyObject reference counting garbage collector and unless faster CPython creates a JIT component, Rython won't have one either. Additionally Ryhon won't have the must have killer libraries like numpy so it's a moot point if my project turns out to be dramatically faster. To sum things up, I've been retired for over a decade so I have plenty of free time. Initially I thought I might invest time into becoming a core python developer but looking into it further, all I will say is that doesn't feel like a very appealing use of my time. On Thu, Jun 23, 2022 at 9:42 AM Avi Gross wrote: David, I am curious why you are undertaking the effort to take a language already decades old and showing signs of being a tad rusty into a language that suggests further oxidation. More seriously, I am interested in what this can gain and the intended user base. I studied Rust for a while and it has it's features but have had no opportunity to use it. Is it expected to make a faster version of Python, or enable better connections to libraries and so on? What I mean is that if you are planning on making it pass all tests for python functionality, are you also adding unique features or ... ? My preference is to have names that fully include what they are about. So the name "python" would be left intact rather than mangled, even if the name itself happens to be totally meaningless. So may I suggest something like """rustic-python""" ? -Original Message- From: David J W To: python-list@python.org Sent: Thu, Jun 23, 2022 10:29 am Subject: Re: "CPython" >> Let's say they reimplement "reference python" CPython in Rust. What is >> better? Change the "reference python" CPython name to RPython, for >> example, or let it as CPython? >The C implementation would still be called CPython, and the new >implementation might be called RPython, or RustyPython, or whatever. >The names are independent of which one is currently blessed as the >reference implementation. I am at the pre planning stages of making a Rust implementation of the Python virtual machine and to avoid ambiguity I've been working with Rython as the name. I tried looking for a Monty Python themed name but the good ones seem to be taken. Otherwise as for a timeline, solo I figure it's going to take me a couple years to get something that actually passes cpython's python unit-tests. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
David, I am curious why you are undertaking the effort to take a language already decades old and showing signs of being a tad rusty into a language that suggests further oxidation. More seriously, I am interested in what this can gain and the intended user base. I studied Rust for a while and it has it's features but have had no opportunity to use it. Is it expected to make a faster version of Python, or enable better connections to libraries and so on? What I mean is that if you are planning on making it pass all tests for python functionality, are you also adding unique features or ... ? My preference is to have names that fully include what they are about. So the name "python" would be left intact rather than mangled, even if the name itself happens to be totally meaningless. So may I suggest something like """rustic-python""" ? -Original Message- From: David J W To: python-list@python.org Sent: Thu, Jun 23, 2022 10:29 am Subject: Re: "CPython" >> Let's say they reimplement "reference python" CPython in Rust. What is >> better? Change the "reference python" CPython name to RPython, for >> example, or let it as CPython? >The C implementation would still be called CPython, and the new >implementation might be called RPython, or RustyPython, or whatever. >The names are independent of which one is currently blessed as the >reference implementation. I am at the pre planning stages of making a Rust implementation of the Python virtual machine and to avoid ambiguity I've been working with Rython as the name. I tried looking for a Monty Python themed name but the good ones seem to be taken. Otherwise as for a timeline, solo I figure it's going to take me a couple years to get something that actually passes cpython's python unit-tests. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
If we want to be humorous, RPython would obviously either be written in R, which really is not designed well for such purposes, or would be some kind of synthesis that already exists that allows you to run R and python code interchangeably on sort of shared data that I sometimes do in RSTUDIO. I like the way you think Greg. I did not consider how the ++ in C++ is a bit like stuttering and since python also starts with a P the effect would be something like C-p-p-python. My problem with that idea is, believe it or not, that it is too negative. What you meant to be seen as a dash is a minus sign to me. And both C and C++ not only have both a pre and post autoincrement variable using ++x and x++, they also have autodecrement operators using a minus sign such as --x and x-- and it can get pretty weird trying to figure out if some code is legal, let alone what it does, without parentheses. I mean what the heck does this do? y = x++-++x The truth is that although I remember Bjarne trying to figure out a good name for his somewhat improved language and his choice of C++ rather than D or some other gimmick, you could argue he also removed a bit from C. But who would call a language C-- ?? Back to serious. This discussion is more about names but is it? Some of the implementations of Python are not just written in some computer language but also in a sort of environment. Arguably some core of functionality has to be pretty much consistent to the language definition. But each may add interesting twists on its own and that can include the ability to easily link in code and libraries written in that language or used in that environment. You can have different supersets of a language. And it can impact where you might use the language as one reason people may not understand for using C is that a compiler was available for just about anywhere that either ran in that environment or could be run on another to produce lower-level code to copy to it. It was also possible to embed code in C that was evaluated differently in each environment for some level of fine-tuning. Some of that may eventually have been true for other implementations but I suspect not for some deliberately designed to fit what one party wants and with no care that it be shared elsewhere especially as the C version was already available there. Or am I wrong? After all, others who kept improving C thought the ++ concept was best removed! -Original Message- From: Greg Ewing To: python-list@python.org Sent: Tue, Jun 21, 2022 3:53 am Subject: Re: "CPython" On 21/06/22 2:56 pm, Paulo da Silva wrote: > Let's say they reimplement "reference python" CPython in Rust. What is > better? Change the "reference python" CPython name to RPython, for > example, or let it as CPython? The C implementation would still be called CPython, and the new implementation might be called RPython, or RustyPython, or whatever. The names are independent of which one is currently blessed as the reference implementation. Although if it were called RPython, no doubt a new debate would flare up over whether the "R" stands for "Rust" or "Reference"... -- Greg -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: "CPython"
This leads to the extremely important question of what would an implementation of Python, written completely in C++, be called? C++Python CPython++ C+Python+ DPython SeaPython? SeeSeaSiPython I don't even want to think fo what sound a C# Python would make. OK, my apologies to all. Being an interpreted language, it makes sense for a good part of the interpreter to include parts made in other languages and also add-on libraries in even older languages like FORTRAN. Quite a few languages, including some like R, are also partially based on C in similar ways. -Original Message- From: Paulo da Silva To: python-list@python.org Sent: Mon, Jun 20, 2022 8:53 pm Subject: Re: "CPython" Às 20:01 de 20/06/22, Paulo da Silva escreveu: > Às 18:19 de 20/06/22, Stefan Ram escreveu: >> The same personality traits that make people react >> to troll postings might make them spread unconfirmed >> ideas about the meaning of "C" in "CPython". >> >> The /core/ of CPython is written in C. >> >> CPython is the /canonical/ implementation of Python. >> >> The "C" in "CPython" stands for C. >> >> > > Not so "unconfirmed"! > Look at this article, I recently read: > https://www.analyticsinsight.net/cpython-to-step-over-javascript-in-developing-web-applications/ > > > > There is a sentence in ther that begins with "CPython, short for Core > Python, a reference implementation that other Python distributions are > derived from, ...". > > Anyway, I wrote "IMHO". > > Do you have any credible reference to your assertion "The "C" in > "CPython" stands for C."? > > Thank you. Well ... I read the responses and they are not touching the point! I just answered, with my opinion based on articles I have read in the past. Certainly I could not be sure. That's why I responded as an opinion (IMHO) and not as an assertion. Stefan Ram responded with a, at least, not very polite post. That's why I needed to somehow "defend" why I posted that response, and, BTW, trying to learn why he said that the C in CPython means "written in C". I still find very strange, to not say weird, that a compiler or interpreter has a name based in the language it was written. But, again, is just my opinion and nothing more. I rest my case. Thank you all. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: fill out bulletins
I wish this discussion was simplified. It sounds to me like what is wanted is a way to PRINT a filled-out form using some dynamic text that fits over designated slots in the data. It is not that different from many other tasks where you overlay some graphics with text. You need a decent version of the graphics with enough pixels and the right size using something as simple as a scanner at higher resolution OR getting someone to give you a template in a form you can use. Using a ruler or other tools, map out the rectangles you are going to be printing text in. Use your own algorithms to take each snippet of text you want to print and do wrapping and resizing of the font or whatever it takes to make it fit. Finally, overlay the text snippets on the image. Then PRINT it on a decent printer. From what I hear, you seem not to need the original text anymore, but nothing stops your program with storing bits you will use again later and reusing them, or having a log of the original text used and so on. Plan B might be to COPY your image on a ream of paper and then replace the paper in the printer so the next print writes on top of it. Create a template with just text that when printed will happen to write over the same spots. As to what tools you can use, there are many to choose from. You asked on a Python list so you may want some of the Python Graphics utilities. In R, I might use ggplot which lets me set a background layer then draw objects above it at various offsets, as one of many things. I know many businesses do things like this all the time such as having printers loaded with checks to be printed, or inserting an envelope into a slot on the printer to have the name and address printedin the right places. -Original Message- From: Peter Pearson To: python-list@python.org Sent: Tue, Jun 14, 2022 9:28 am Subject: Re: fill out bulletins On Tue, 14 Jun 2022 00:41:07 +0200, jak wrote: [snip] > > If you are interested in seeing what I called "post office bulletin" > (English is not my language and I don't know the name, sorry), you can > find a sample pdf (fillable) but it works badly here: > > https://www.guardiacostiera.gov.it/venezia/Documents/Bollettino%20MOD.%20TD123.pdf Are these "post office bulletins" always PDFs? -- To email me, substitute nowhere->runbox, invalid->com. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Function to Print a nicely formatted Dictionary or List?
Dave, Despite your other programming knowledge, I suspect you think this is the forum where people come to be tutored. Try here: https://mail.python.org/mailman/listinfo/tutor Yes, there are plenty of pretty printers available and you can build your own function fairly easily. A module like pprint may have what you want in pprint.pprint() but you can write a function for yourself that takes a dictionary and loops through items and prints them one per line and, if you feel like it, also prints how many items there are and your own custom touches such as doing them alphabetically. Consider using a search engine before posting. Throw in a few words like "python pretty print dictionary function" and refine that if it does not get you immediate results. It is free and easy and does not waste time for so many others who already know or don't care. And consider reading a few books perhaps designed to teach python to people with some programming experience or taking a course on something like COURSERA as a part of your learning process and not depending on volunteers so much. Much of what you are asking is covered in fairly beginner and intermediate such books/courses. I think I am now going to ignore messages from you for a while. Signal to noise ratio ... -Original Message- From: Dave To: python-list@python.org Sent: Thu, Jun 9, 2022 6:43 am Subject: Function to Print a nicely formatted Dictionary or List? Hi, Before I write my own I wondering if anyone knows of a function that will print a nicely formatted dictionary? By nicely formatted I mean not all on one line! Cheers Dave -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test characters of a string
Dave, Sometimes a task is done faster by NOT programming anything in any language! Not only have you spent a lot of your own time but many dozens of messages here have dragged in others, who gain nothing ;-) The domain you are operating in seems to have lots of variants in how the titles are stored as names and you keep finding new variants. Yes, if your goal is to use this as a way to learn more in general about Python, clearly it may meet that goal! Contrary to what I (and some others) said earlier, it may be time to consider regular expressions and other heavier artillery! LOL! I do not plan on sticking with your twists and turns but will quickly address your variants. Sides that come in two's like records are presumably limited to using A and B in your example. But multiple disks connected can mean the digit(s) following can have double digits or even more. Your python code may have to contain lots of functions you create that match some pattern and return some value and perhaps other functions that know how to convert from that format to a common canonical format of your own so they can be compared. Your main code may need to try them in various sequences till it finds a match and so on. But when you are done, in what format do you save them? The original or your minimal? Still confusing to me, as someone who does not give a darn, is the reality that many songs may have the same name but be different as in a song from Sinatra when he was young and a later recording with a different orchestra or by a Sinatra imitator. They may all be titled something like "New York, New York" or "NEW YORK -- NEW YORK" which your algorithm folds into the same characters. So I am guessing you also need to access other data about the person who sings it or what year it was released to make comparisons. At some point you may want to create or borrow some sort of class/object that encapsulates your data as well as methods that let you do things like make a canonical version of the Title and then a way to ask if Object A is reasonably equal to object B might happen if you define a function/method of __eq__ for that class. It might take you years and need periodic refining as you encounter ever more varied ways people have chosen to label their music, but so what? LOL! Humor or sarcasm aside, your incremental exploratory method reminds me why it is a good idea to first scope out the outlines of your problem space and make some key decisions and write out a fairly detailed set of requirements before seriously making more than prototypes. You might get very different help from people if they understood that your first request was far from complete but only one of many that may better be worked on some other way. And I wonder if you did any search of the internet to see if anyone had done anything similar in Python (or another language) that may handle parts of what you need before asking here. I note lots of people who come with what they consider a good programming background have to adjust to aspects of languages like python as what they know is in some ways wrong or inadequate in a new environment. -Original Message- From: Dave To: python-list@python.org Sent: Thu, Jun 9, 2022 2:50 am Subject: Re: How to test characters of a string Hi, I’ve found you also need to take care of multiple disk CD releases. These have a format of “1-01 Track Name” “2-02 Trackl Name" Meaning Disk 1 Track1, Disk 2, Track 2. Also A and B Sides (from Vinyl LPs) “A1-Track Name” “B2-Track Name” Side A, Track 1, etc. Cheers Dave > On 8 Jun 2022, at 19:36, Dennis Lee Bieber wrote: > > On Wed, 8 Jun 2022 01:53:26 + (UTC), Avi Gross > declaimed the following: > > >> >> So is it necessary to insist on an exact pattern of two digits followed by a >> space? >> >> >> That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways >> to Say Goodbye", "99 Ways to Die" >> >> It looks to me like you need to compare TWICE just in case. If it matches in >> the original (perhaps with some normalization of case and whitespace, fine. >> If not will they match if one or both have something to remove as a prefix >> such as "02 ". And if you are comparing items where the same song is in two >> different numeric sequences on different disks, ... > > I suspect the OP really needs to extract the /track number/ from the > ID3 information, and (converting to a 2digit formatted string) see if the > file name begins with that track number... The format of the those > filenames appear to be those generated by some software when ripping CDs to > MP3s -- for example: > > -=-=- > c:\Music\Roger Miller\All Time Greatest Hits>dir > Volume in drive C is OS > Vo
Re: How to replace characters in a string?
Dave,
Your goal is to compare titles and there can be endless replacements needed if
you allow the text to contain anything but ASCII.
Have you considered stripping out things instead? I mean remove lots of stuff
that is not ASCII in the first place and perhaps also remove lots of extra
punctuation likesingle quotes or question marks or redundant white space and
compare the sort of skeletons of the two?
And even if that fails, could you have a measure of how different they are and
tolerate if they were say off by one letter albeit "My desert" matching "My
Dessert" might not be a valid match with one being a song about an arid
environment and the other about food you don't need!
Your seemingly simple need can expand into a fairly complex project. There may
be many ideas on how to deal with it but not anything perfect enough to catch
all cases as even a trained human may have to make decisions at times and not
match what other humans do. We have examples like the TV show "NUMB3RS" that
used a perfectly valid digit 3 to stand for an "E" but yet is often written
when I look it up as NUMBERS. You have obvious cases where titles of songs may
contain composite symbols like "œ" which will not compare to one where it is
written out as "oe" so the idea of comparing is quite complex and the best you
might do is heuristic.
UNICODE has many symbols that are almost the same or even look the same or
maybe in one font versus another. There are libraries of functions that allow
some kinds of comparisons or conversions that you could look into but the gain
for you may not be worth it. Nothing stops a person from naming a song any way
they want and I speak many languages and often see a song re-titled in the
local language and using the local alphabet mixed often with another.
Your original question is perhaps now many questions, depending on what you
choose. You started by wanting to know how to compare and it is moving on to
how to delete parts or make substitutions or use regular expressions and it can
get worse. You can, for example, take a string and identify the words within it
and create a regular expression that inserts sequences between the words that
match any zero or one or more non-word characters such as spaces, tabs,
punctuation or non-ASCII, so that song titles with the same words in a sequence
match no matter what is between them. The possibilities are endless but
consider some of the techniques that are used by some programs that parse text
and suggest alternate spellings or even programs like Google Translate that
can take a sentence and then suggest you may mean a slightly altered sentence
with one word changed to fit better.
You need to decide what you want to deal with and what will be mis-classified
by your program. Some of us have suggested folding the case of the words but
that means asong about a dark skinned person in Poland called "Black Polish"
would match a song about keeping your shoes dark with "black polish" so I keep
repeating it is very hard or frankly impossible, to catch every case I can
imagine and the many I can't!
But the emphasis here is not your overall problem. It is about whether and how
the computer language called python, and perhaps some add-on modules, can be
used to solve each smaller need such as recognizing a pattern or replacing
text. It can do quite a bit but only when the specification of the problem is
exact.
-Original Message-
From: Dave
To: python-list@python.org
Sent: Wed, Jun 8, 2022 5:09 am
Subject: Re: How to replace characters in a string?
Hi,
Thanks for this!
So, is there a copy function/method that returns a MutableString like in
objective-C? I’ve solved this problems before in a number of languages like
Objective-C and AppleScript.
Basically there is a set of common characters that need “normalizing” and I
have a method that replaces them in a string, so:
myString = [myString normalizeCharacters];
Would return a new string with all the “common” replacements applied.
Since the following gives an error :
myString = 'Hello'
myNewstring = myString.replace(myString,'e','a’)
TypeError: 'str' object cannot be interpreted as an integer
I can’t see of a way to do this in Python?
All the Best
Dave
> On 8 Jun 2022, at 10:14, Chris Angelico wrote:
>
> On Wed, 8 Jun 2022 at 18:12, Dave wrote:
>
>> I tried the but it doesn’t seem to work?
>> myCompareFile1 = ascii(myTitleName)
>> myCompareFile1.replace("\u2019", "'")
>
> Strings in Python are immutable. When you call ascii(), you get back a
> new string, but it's one that has actual backslashes and such in it.
> (You probably don't need this step, other than for debugging; check
> the string by printing out the ASCII version of it, but stick to the
> original for actual processing.) The same is true of the replace()
> method; it doesn't change the string, it returns a new string.
>
word = "spam"
print(word.replace("sp", "h"))
>
Re: How to test characters of a string
Amazing how some people bring out the heavy artillery, first! LOL! If the question was how to remove any initial digits and perhaps whitespace in a string, it is fairly easy to do without any functions to test if there are digits before the title. I mean look at initial characters and move forward if it is between '0' and '9' or a space. Duh! Sure, a regular expression that matches anything following a run of digits and whitespace and before a ".MPG" or the end of the entry will be easy to extract and compare after removing any left/right whitespace in both things being compared and coercing both to the same case. But the solution may be doomed to failure when it sees things like: "100 Letters" "1+1" "10,000 hours" "1 Trillion Dollar$" "2,000 Light Years From Home" So is it necessary to insist on an exact pattern of two digits followed by a space? That would fail on "44 Minutes", "40 Oz. Dream", "50 Mission Cap", "50 Ways to Say Goodbye", "99 Ways to Die" It looks to me like you need to compare TWICE just in case. If it matches in the original (perhaps with some normalization of case and whitespace, fine. If not will they match if one or both have something to remove as a prefix such as "02 ". And if you are comparing items where the same song is in two different numeric sequences on different disks, ... -Original Message- From: Christian Gollwitzer To: python-list@python.org Sent: Tue, Jun 7, 2022 6:01 pm Subject: Re: How to test characters of a string Am 07.06.22 um 21:56 schrieb Dave: > It depends on the language I’m using, in Objective C, I’d use isNumeric, just > wanted to know what the equivalent is in Python. > Your problem is also a typical case for regular expressions. You can create an expression for "starts with any number of digits plus optional whitespace" and then replace this with nothing: > chris@linux-tb9f:~> ipython > Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] > Type 'copyright', 'credits' or 'license' for more information > IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help. > > In [1]: import re > > > > In [2]: s='05 Trinket' > > > > In [3]: re.sub(r'^\d+\s*', '', s) > > > Out[3]: 'Trinket' > If it doesn't match, it will do nothing: > In [4]: s='Es geht los' > > > > In [5]: re.sub(r'^\d+\s*', '', s) > > > Out[5]: 'Es geht los' Some people on this list don't like regexes but for tasks like this they are made and working well. ^ is "starts with" \d is any digit \s is any space + is at least one * is nothing or one of Christian -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: oop issue
invest_crypto.client_list.append(self)
I am wondering about the phrasing above.
When you are in the dunder init function, you normally create and change items
in YOURSELF so why is your code not changing self.crypto_client_list?
And what are you appending to before ever creating it? Would it kill you to
create some form of container in the main class definition initialized
appropriately to some variation of empty?
I won't claim to fully understand what the code wants to do. Adding yourself to
the list of clients may make sense to you but if you understand object oriented
programming, you may have an inkling that objects need to be CREATED somewhere
before they can be used. Most of your code looks like it is DEFINING an object.
The last few lines try to access a method in an object that has never been
instantiated. Yes, you do have a way to store methods in a class and call them
without any objects but this is not a case like that.
You need something like "myobj = invest_crypto(args)" and then the rest of your
code can do changes and calculations and perhaps create other similar or
different objects. You seem to be using a method that reads in a file and dumps
a version of the contents and that might work if you did a line like this next:
"myobj.access_client_details()" albeit not how I would name it or do it.
And, of course, your print() again names the class, not an instance of a class.
Your code wraps in a few places and I wonder if it contains errors as in this
set of lines:
return f"('{self.name}', '{self.surname}', '{self.amount_Deposited}',
'{self.amount_to_transfer}')"
Is that really how you think you set up a formatted string?
Even if you get that working, how does the print statement know what to do with
a list of objects?
Some might design a method you can call to print the contents of a list which
would loop over the list. But is there ever more than one client (yourself) in
the list?
The above rambling is just reflecting my opinion that you have not learned
enough or thought it through and may even be copying and modifying various
snippets of code perhaps from places where it works to a place that might be
better designed from scratch.
Have fun. As someone else mentioned, smaller more focused examples may work
better to get you up to speed, but then again, we often find out someone is
given a homework assignment ...
-Original Message-
From: Tola Oj
To: python-list@python.org
Sent: Mon, May 23, 2022 4:54 pm
Subject: oop issue
i just finished learning oop as a beginner and trying to practice with it
but i ran into this typeerror issue, help please.
Traceback (most recent call last):
File
"c:\Users\ojomo\OneDrive\Desktop\myexcel\oop_learn.py\myExperiment.py\mainMain.py",
line 36, in
print(invest_crypto.client_list)
TypeError: invest_crypto.__repr__() missing 1 required positional argument:
'self'
this is my code below:
import csv
class invest_crypto:
crypto_current_rate = 0.05
client_list = []
def __init__(self, name, surname, amount_Deposited, amount_to_transfer):
self.name = name
self.surname = surname
self.amount_Deposited = amount_Deposited
self.amount_to_transfer = amount_to_transfer
invest_crypto.client_list.append(self)
def calculate_customer_transfer(self):
self.customer_transfer = (self.crypto_current_rate * self.
amount_Deposited) + self.amount_Deposited
return self.customer_transfer
@classmethod
def access_client_details(cls):
with open('C:\\Users\\ojomo\\OneDrive\\Desktop\\myexcel\\
oop_learn.py\\myExperiment.py\\clientDetails.csv', 'r' ) as f:
reader = csv.DictReader(f)
clientDetails = list(reader)
for item in clientDetails:
invest_crypto(
name=item.get('name'),
surname=item.get('surname'),
amount_Deposited=item.get('amount_deposited'),
amount_to_transfer=item.get('amount_to_transfer')
)
@staticmethod
def __repr__(self):
return f"('{self.name}', '{self.surname}', '{self.amount_Deposited}',
'{self.amount_to_transfer}')"
invest_crypto.access_client_details()
print(invest_crypto.client_list)
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: .0 in name
You left out 3CPO, Dave. Names with numerals are not unreasonable in some circumstances. But programming languages are not a full spectrum of real life. They can have reserved words too so you may not be able to use "while" and if you choose to create a function that masks another, you cannot complain that you assumed the computer would be smart enough to figure out which one you wanted. Yes, some languages encourage multiple functions with the same name but different signatures. Bottom line is each has RULES and some serious SUGGESTIONS. It is what it is, not what you want it to be. But although starting with a numeral is verboten for variable names, may I suggest that a relatively invisible character can make a name like _3CPO that will be allowed. But be careful as Python programs often have conventions on use of the underscore and don't even think about a dundering name like __init__ ... -Original Message- From: dn To: python-list@python.org Sent: Sat, May 14, 2022 12:33 am Subject: Re: .0 in name This is not what @Avi menat by "silly variable names" but: 3D_position 2nd_floor_area 3M_PostIt_size 3rd_degree_polynomial 360_degree_view 12_hours_later and ??? 2_fast_2_furious -- Regards, =dn -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: .0 in name
Boy do I hate when I see my code mangled by the stupid AOL mailer.
Not that anyone cares, but the code should be read with each line starting with
the "> " prompt.
If I leave lots of blank lines, it may work, but as the illustration is not in
python, I will now remove the prompts:
`5x^2 + 2.3x` <- 666
`+-+-+` <- 1000
1 + 2 * `5x^2 + 2.3x` + `+-+-+`
output: 2333
There are rarely good reasons for such silly variable names but as long as you
let it know to leave the quoted regions alone when parsing and look them us as
exact entries in various environments, it works fine.
To add to what others already wrote, many languages have serious requirements
you need to be aware of and not assume otherwise. Some allow underscores in
names and may limit that in the first part or may in some cases suggest or
require it. Some have rules about whether a variable of one kind should start
with an uppercase letter. Some allow periods in names although an initial
period may make it invisible for some purposes. And, some newer languages allow
all kinds of UNICODE characters and perhaps even some that can be seen as
numeric but aren't exactly 0-9.
① ② ③
... ❽ ❽
-Original Message-----
From: Avi Gross via Python-list
To: python-list@python.org
Sent: Fri, May 13, 2022 6:02 pm
Subject: Re: .0 in name
Bryan,
As has been pointed out, it is very common in possibly all programming
languages to not allow digits at the start of many identifiers. It makes it
hard to parse for numbers which tend to start with digits. Some languages even
have special rules on not starting a number with a zero unless you mean for it
to be seen as octal (or 0x for hexadecimal) and many other rules exist.
There are languages where 12x means 12*x so even the lack of an operator ...
There are exceptions that often are not really exceptions. You can use all
kinds of unusual variables in some quoted context. It is valid (albeit not
encouraged) to use backquoted
The following is perfectly allowed in R:
> `5x^2 + 2.3x` <- 666 > `+-+-+` <- 1000 > 1 + 2 * `5x^2 + 2.3x` + `+-+-+` [1]
> 2333
And there are often issued when you do things like create the name of a column
of data in a data.frame with embedded spaces and other anomalies requiring
special handling.
So why you wonder where it is documented that variables cannot be what you feel
like is a bit puzzling!
-Original Message-
From: bryangan41
To: python-list@python.org
Sent: Fri, May 13, 2022 12:47 pm
Subject: .0 in name
May I know (1) why can the name start with a number?(2) where in the doc is
it?!>>> import pdb>>> pdb.run('(a for a in "")')> (1)()(Pdb)
s--Call--> (1)()(Pdb) a.0 = (Pdb) c>>>Sent from Samsung tablet.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: .0 in name
Bryan,
As has been pointed out, it is very common in possibly all programming
languages to not allow digits at the start of many identifiers. It makes it
hard to parse for numbers which tend to start with digits. Some languages even
have special rules on not starting a number with a zero unless you mean for it
to be seen as octal (or 0x for hexadecimal) and many other rules exist.
There are languages where 12x means 12*x so even the lack of an operator ...
There are exceptions that often are not really exceptions. You can use all
kinds of unusual variables in some quoted context. It is valid (albeit not
encouraged) to use backquoted
The following is perfectly allowed in R:
> `5x^2 + 2.3x` <- 666 > `+-+-+` <- 1000 > 1 + 2 * `5x^2 + 2.3x` + `+-+-+` [1]
> 2333
And there are often issued when you do things like create the name of a column
of data in a data.frame with embedded spaces and other anomalies requiring
special handling.
So why you wonder where it is documented that variables cannot be what you feel
like is a bit puzzling!
-Original Message-
From: bryangan41
To: python-list@python.org
Sent: Fri, May 13, 2022 12:47 pm
Subject: .0 in name
May I know (1) why can the name start with a number?(2) where in the doc is
it?!>>> import pdb>>> pdb.run('(a for a in "")')> (1)()(Pdb)
s--Call--> (1)()(Pdb) a.0 = (Pdb) c>>>Sent from Samsung tablet.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: tail
This seems to be a regular refrain where someone wants something as STANDARD in a programming language or environment and others want to keep it lean and mean or do not see THIS suggestion as particularly important or useful. Looking at the end of something is extremely common. Packages like numpy/pandas in Python often provide functions with names like head or tail as do other languages where data structures with names like data.frame are commonly used. These structures are in some way indexed to make it easy to jump towards the end. Text files are not. Efficiency aside, a 3-year-old (well, certainly a 30 year old) can cobble together a function that takes a filename assumed to be textual and reads the file into some data structure that stores the lines of the file and so it can be indexed by line number and also report the index of the final line. The data structure can be a list of lists or a dictionary with line numbers as keys or a numpy ... So the need for this functionality seems obvious but then what about someone who wants a bunch of random lines from a file? Need we satisfy their wish to pick random offsets from the file and get the line in which the offset is in middle of or the one about to start? Would that even be random if line lengths vary? Text files were never designed to be used efficiently except for reading and writing and certainly not for something like sorting. Again, generally you can read in the darn file and perform the operation and free up whatever memory you do not need. If you have huge files, fine, but then why make a special function be part of the default setup if it is rarely used? Why not put it in a module/package called BigFileBatches alongside other functions useful to do things in batches? Call that when needed but for smaller files, KISS. -Original Message- From: Dennis Lee Bieber To: python-list@python.org Sent: Wed, May 11, 2022 6:15 pm Subject: Re: tail On Thu, 12 May 2022 06:07:18 +1000, Chris Angelico declaimed the following: >I don't understand why this wants to be in the standard library. > Especially as any Linux distribution probably includes the compiled "tail" command, so this would only be of use on Windows. Under recent Windows, one has an equivalent to "tail" IFF using PowerShell rather than the "DOS" shell. https://www.middlewareinventory.com/blog/powershell-tail-file-windows-tail-command/ or install a Windows binary equivalent http://tailforwin32.sourceforge.net/ -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
Just FYI, UNIX had a bunch of utilities that could emulate a vanilla version of tail on a command line. You can use sed, awk and quite a few others to simply show line N to the end of a file or other variations. Of course the way many things were done back then had less focus on efficiency than how to stepwise make changes in a pipeline so reading from the beginning to end was not an issue. -Original Message- From: Marco Sulla To: Chris Angelico Cc: python-list@python.org Sent: Wed, May 11, 2022 5:27 pm Subject: Re: tail On Wed, 11 May 2022 at 22:09, Chris Angelico wrote: > > Have you actually checked those three, or do you merely suppose them to be > true? I only suppose, as I said. I should do some benchmark and some other tests, and, frankly, I don't want to. I don't want to because I'm quite sure the implementation is fast, since it reads by chunks and cache them. I'm not sure it's 100% free of bugs, but the concept is very simple, since it simply mimics the *nix tail, so it should be reliable. > > > I'd very much like to see a CPython implementation of that function. It > > could be a method of a file object opened in binary mode, and *only* in > > binary mode. > > > > What do you think about it? > > Still not necessary. You can simply have it in your own toolkit. Why > should it be part of the core language? Why not? > How much benefit would it be > to anyone else? I suppose that every programmer, at least one time in its life, did a tail. > All the same assumptions are still there, so it still > isn't general It's general. It mimics the *nix tail. I can't think of a more general way to implement a tail. > I don't understand why this wants to be in the standard library. Well, the answer is really simple: I needed it and if I found it in the stdlib, I used it instead of writing the first horrible function. Furthermore, tail is such a useful tool that I suppose many others are interested, based on this quick Google search: https://www.google.com/search?q=python+tail A question on Stackoverflow really much voted, many other Stackoverflow questions, a package that seems to exactly do the same thing, that is mimic *nix tail, and a blog post about how to tail in Python. Furthermore, if you search python tail pypi, you can find a bunch of other packages: https://www.google.com/search?q=python+tail+pypi It seems the subject is quite popular, and I can't imagine otherwise. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: tail
Marco, I think it was made clear from the start that "text" files in the classic sense have no random access method at any higher level than reading a byte at some offset from the beginning of the file, or back from the end when it has not grown. The obvious fact is that most of the time the lines are not of fixed widths and you have heard about multiple byte encodings and how the ends of lines can vary. When files get long enough that just reading them from the start as a whole, or even in chunks, gets too expensive, some people might consider some other method. Log files can go on for years so it is not uncommon to start a new one periodically and have a folder with many of them in some order. To get the last few lines simply means finding the last file and reading it, or if it is too short, getting the penultimate one too. And obviously a database or other structure might work better which might make each "line" a record and index them. But there are ways to create your own data that get around this such as using an encoding with a large but fixed width for every character albeit you need more storage space. But if the goal is a general purpose tool, internationalization from ASCII has created a challenge for lots of such tools. -Original Message- From: Marco Sulla To: Dennis Lee Bieber Cc: python-list@python.org Sent: Sat, May 7, 2022 9:21 am Subject: Re: tail On Sat, 7 May 2022 at 01:03, Dennis Lee Bieber wrote: > > Windows also uses for the EOL marker, but Python's I/O system > condenses that to just internally (for TEXT mode) -- so using the > length of a string so read to compute a file position may be off-by-one for > each EOL in the string. So there's no way to reliably read lines in reverse in text mode using seek and read, but the only option is readlines? -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
This topic has rapidly shifted over way beyond Python even as the original person has not returned to participate. There are many ways to teach anything and since the classical method was to learn in person from someone using mainly sound or pantomime, it has hung on. Even with the existence of writing, books were made one at a time and were rare. In more recent times, the norm shifted gradually from lectures to individuals and groups to include textbooks and illustrations and eventually recordings or PowerPoint Slides with animation. Realistically, learning some things on your own is easy enough but for many people and many subjects, you need interaction, reinforcement and more. We have college professors who repeat almost the same identical lectures for years to various audiences and also take few or no questions. they might as well be recorded and find something better to do. But how do you learn French just from a book when it is completely not obvious how to pronounce anything given the weird spelling and grammar rules? How do you know if the dialect you use swallows some sounds or stretches them out a bit? For this you need to hear and perhaps see native speakers and correlate those sounds to the written words and learn to recognize and make them a habit. Even better, you often want someone to listen to what you try to say and respond and help guide you. Many Computer topics have an interesting side in that access to a computer running whatever you are learning can give you much experience and guidance as you can try various things and see how they work. Written text alone may be enough to learn what is special about a language and a set of problems to work on (or your own exploration) may be able to replace much of human interaction. You can look at learning systems such as COURSERA where they often break a "class" into parts that can include often shorter video clips often with subtitles or transcripts alongside it, as well as various kinds of printed material including tests and assignments and even ways (in some programming courses) to write small programs that are evaluated immediately by running them through the language program, or by having others (sometimes fellow students) grade them and return the results to you. There are many ideas out there how to learn. One of the worst is huge lecture halls with no rewind ... But text-only learning tools vary quite a bit and some of the better ones do not just throw facts at you but stop periodically and give you an overview of the goals and maybe add a touch of history that provides context on why some innovation was such a big improvement over what had been done and help you pronounce things when it is not obvious by saying that many people say a function name to rhyme with this or ... I used to hate Math textbooks that used every imaginable symbol and assumed you knew how to say every Greek letter and script L and integral symbol and an assortment of braces and brackets in various sizes and much more. It is hard to memorize formulas where you call lots of items by the name of "squiggle"! Python currently sticks largely to using standard ASCII characters so it has fewer issues to deal with. For people who are not native English speakers, though, some things may not be intuitively obvious, let alone pronounceable. I suspect for some purposes, a few lectures to listen to might help if well-designed. But I noticed how in Julia, they allow all kinds of symbols but also provide a way to make them fairly easily. Still their use of an actual lower-case epsilon as a synonym for "in" is an example of how teaching Julia may need more thantext for some people. It uses lots of unusual symbols for operators too thatare often familiar to mathematicians and hardly anyone else. for i ∈ 1:10 -Original Message- From: 2qdxy4rzwzuui...@potatochowder.com To: python-list@python.org Sent: Fri, May 6, 2022 8:56 am Subject: Re: Python/New/Learn On 2022-05-05 at 16:51:49 -0700, Grant Edwards wrote: > On 2022-05-05, Mats Wichmann wrote: > > > Without having any data at all on it, just my impressions, more > > people these days learn from in-person or video experiences. > > I've always been utterly baffled by video tutorials for > programming. There must be people who prefer that format, but it seems > like absolutely the worst possible option for me. You can't cut/paste > snippets from the examples. You have to constantly pause them so you > can try out examples. Sometimes it's not even easy to read the > examples. Perhaps if there was an accompanying web page or PDF... +1 (maybe more), except that an accompanying web page or PDF only solves the problem of copying/pasting examples badly, at the expense of the cognitive load to keep track of one more thing (because it's highly unlikely that the web page or PDF tracks the video "automatically"). As far as easy-to-read examples go, writing them down doesn't
Re: Python/New/Learn
Before more people reply to this user, I note I have not seen them reply back to the list about any questions or comments others have taken the time to provide. My warning bells go off when I see patterns and there was a similar request from another gmail account to an R language forum I am also on. They wanted help in learning programming (especially R) and claimed not to have any source to study. As we keep pointing out, you can trivially find sources including many free ones. So I wonder if there is point being sucked in by one or more people who don't even continue a conversation and perhaps are not even listening but just playing us to make us waste time. Or, maybe they are asking on multiple places to decide which to choose and are not saying so. Am I paranoid? Nah! But yes, a bit wary. I get so many kinds of SPAM in mail and phone calls and lately keep getting calls asking if I want to sell my house ... -Original Message- From: Schachner, Joseph To: Patrick 0511 ; python-list@python.org Sent: Thu, May 5, 2022 12:04 pm Subject: RE: Python/New/Learn Buy the book "Python 101" and do the examples. When you're done with that buy the book "Python 201" and study it. There is much more than is in both those books that you could learn about Python, but that's a very good way to start. --- Joseph S. Teledyne Confidential; Commercially Sensitive Business Data -Original Message- From: Patrick 0511 Sent: Wednesday, May 4, 2022 9:36 PM To: python-list@python.org Subject: Python/New/Learn Hello, I'm completely new here and don't know anything about python. Can someone tell me how best to start? So what things should I learn first? -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
I agree Chris that the Ukrainian Python Books are daunting as I barely started learning that language now even though my early years were just a few miles away and I might even have relatives still there! But as has been pointed out, suggestions are more helpful if you know a bit more about the one asking the questions or it is too broad. I do not feel qualified to suggest a book to true beginners as I leaned Python many decades after learning so many other computer languages and read a wide variety of books including many that assumed you already knew C or R or other languages, as I did. Someone new to programming may fin some resources including a tutorial handy. Someone who actually wants to use various modules like scikit-learn or pandas, or to solve specific problems, might well want to go straight to other resources or realize they need multiple resources over time. It can be a diversion to get someone to learn the functional programming aspects or even object-oriented if the goal is to do simple procedural things. Python is an extremely rich language and I recall reading a three-volume encyclopedia that took months as the books got pulled away by other readers. So, no Mark Lutz got a bit advanced. Oddly, I am learning Julia now and cannot find a single book in my group of Libraries, not even for the beginner I am not! But finding online resources was easy and if I have any reason to, plenty of books can be bought. It does not work for everybody, but my personal method is to attack issues and problems from multiple directions as each tends to reinforce and add to my knowledge. And I really appreciate when they tell you specifically how the language is different from others you may know so you know when not to expect it to work. A Julia documentation as an example, has a long list of places it is not like Python, or R or C++ and so on. If the person has computer language experience, some such resource might be better than something spending a thousand pages to teach from scratch. But, I am not volunteering to do personal tutoring. I prefer responding to specific focused questions especially after the person seems to have done some searching and research and reading on their own and maybe even shares some code and asks what may be wrong with it or ... And my first question would be why they chose to ask about Python. Was it their choice or required for a course or job or ... Sometimes the answer is to use something else they already know, albeit Python is a very decent language for many uses and well-worth learning even if you know others. -Original Message- From: Chris Angelico To: Avi Gross Cc: python-list@python.org Sent: Wed, May 4, 2022 11:21 pm Subject: Re: Python/New/Learn On Thu, 5 May 2022 at 13:14, Avi Gross wrote: > > Chris, > > It was an extremely open-ended question to a forum where > most of the readers are more advanced, at least I think. > > > My library has oodles of Python Books for free to borrow on paper and > return and I have read many of them. There are various e-books too, and > of course lots of free internet resources including videos and on-line > courses. > > > If he wants simpler books, the web pages pointed here too: > > > https://wiki.python.org/moin/IntroductoryBooks > > > Next time, I won't try to be helpful and brief and just be silent. > Being helpful is great, it's just that being brief can leave it as an incredibly scary-looking list :) If you want to recommend a couple of specific books, I think that would be a lot more helpful. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
Chris, It was an extremely open-ended question to a forum where most of the readers are more advanced, at least I think. My library has oodles of Python Books for free to borrow on paper and return and I have read many of them. There are various e-books too, and of course lots of free internet resources including videos and on-line courses. If he wants simpler books, the web pages pointed here too: https://wiki.python.org/moin/IntroductoryBooks Next time, I won't try to be helpful and brief and just be silent. -Original Message- From: Chris Angelico To: python-list@python.org Sent: Wed, May 4, 2022 11:02 pm Subject: Re: Python/New/Learn On Thu, 5 May 2022 at 12:57, Avi Gross via Python-list wrote: > > https://wiki.python.org/moin/PythonBooks > That's an incredibly daunting list, and not something I'd overly strongly recommend, but yes, if you want to get a dead-tree or e-book to read, there are quite a lot of options available. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Python/New/Learn
https://wiki.python.org/moin/PythonBooks -Original Message- From: Patrick 0511 To: python-list@python.org Sent: Wed, May 4, 2022 9:36 pm Subject: Python/New/Learn Hello, I'm completely new here and don't know anything about python. Can someone tell me how best to start? So what things should I learn first? -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
Yes, Michael, a dictionary is an excellent way to represent a closed set of
transitions which your permutations are.
You examples use numerals but obviously a dictionary will allow transformations
of anything that can be hashed which mostly is items that are not mutable.
Of course for the purposes you may be using these permutations, accessing them
may be done carefully as you need to have nothing else in the dictionary and
must access all the keys and make sure you know which keys have already been
visited from another item.
Some other data structures may work better or faster on smaller examples.
I think you have satisfied my curiosity and your main and only question really
was on suggested wording of a Docstring.
Now if only the Docstring idea was replaced by a Dictionary too! Things like:
Dictstring = {"Purpose": "Text", "Args": "Text", "Return(s)": "Text",
"Optional-Note": "Text", "French version": DocStringFrench}
Too late to seriously change the language now!
-Original Message-
From: Michael F. Stemper
To: python-list@python.org
Sent: Sun, Apr 24, 2022 9:24 am
Subject: Re: Style for docstring
On 23/04/2022 12.43, Avi Gross wrote:
> Given what you added, Michael, your function is part of a larger collection
> of functions and being compatible with the others is a valid consideration.
> Whatever you decide, would ideally be done consistently with all or most of
> them.
> And, of course, it others in the collection also can handle multiple ways to
> specify a permutation, it may be simpler to have each call something like
> as.permutation() that handlesmultiple forms and converts to the one easiest
> for you to use.
> I am not sure that is needed as I suspect the simplest storage is something
> like a list: [0,3,2,4,5,6,7,1,9,8] but could also be shown with each cycle
> as a sub-list or something like anumpy vector or a customized class.
Since you ask, I'm using dictionaries as the internal representation.
If you think about it, a python dictionary *is* a function from one
finite set to another, mathematically. And a (finite) permutation is
a bijection from a (finite) set to itself.
For convenience, the module provides two methods of defining a permutation
other than just entering a dictionary:
>>> import PermGroups as pg
>>> a = {'1':'2', '2':'1', '3':'3'}
>>> b = pg.ParsePerm( '(12)(3)' )
>>> c = pg.ParseDomImg( '123', '213' )
>>> a==b
True
>>> b==c
True
>>>
All of the other functions work on these dictionaries.
I had thought about defining a permutation object, but the conceptual
match between "dict" and "permutation" was too good to discard.
> Clearly if you control the package and how it is used, errors from bad data
> may not be a concern.
An invalidly-constructed permutation will cause an exception, so
the function won't return.
>>> d = {'1':'2', '2':'2', '3':'3'}
>>> pg.ValidateDict(d)
False
>>>
If I was to do it over, I would have named this function something
like IsValidPermutation(), hiding the internal representation as
well as making the function's Boolean nature explicit.
--
Michael F. Stemper
No animals were harmed in the composition of this message.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: tail
I have been getting confused by how many interpretations and conditions for chasing tail people seem to be talking about. A fairly normal task is to want to see just the last N lines of a text-based file. A variant is the "tail -f" command from UNIX that continues to follow a growing file, often into a pipeline for further processing. The variant now being mentioned is a sort of "reverse" that has nothing to do with that kind of "tail" except if the implementation is to read the file backwards. A very straightforward way to reverse a file takes perhaps two lines of Python code by reading forward to fill a list with lines of text then using an index that reverses it. The issues being considered are memory and whether to read the entire file. I would think reading a file forwards in big chunks to be far faster and simpler than various schemes mentioned here for reading it backwards. It only makes sense if the goal is not reversal of all the contents. Also noted is that memory use can be minimized various ways so that only thefinal results are kept around. And if you really want more random access to files that you view as being organized as lines of text with a fixed or maximum width,then storing in some database format, perhaps indexed, may be a way to go. A time stamped log file is a good example. So which problem is really supposed to be solved for the original question? -Original Message- From: Roel Schroeven To: python-list@python.org Sent: Sun, Apr 24, 2022 5:19 am Subject: Re: tail dn schreef op 24/04/2022 om 0:04: > Disagreeing with @Chris in the sense that I use tail very frequently, > and usually in the context of server logs - but I'm talking about the > Linux implementation, not Python code! If I understand Marco correctly, what he want is to read the lines from bottom to top, i.e. tac instead of tail, despite his subject. I use tail very frequently too, but tac is something I almost never use. -- "Peace cannot be kept by force. It can only be achieved through understanding." -- Albert Einstein -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
Given what you added, Michael, your function is part of a larger collection of functions and being compatible with the others is a valid consideration. Whatever you decide, would ideally be done consistently with all or most of them. And, of course, it others in the collection also can handle multiple ways to specify a permutation, it may be simpler to have each call something like as.permutation() that handlesmultiple forms and converts to the one easiest for you to use. I am not sure that is needed as I suspect the simplest storage is something like a list: [0,3,2,4,5,6,7,1,9,8] but could also be shown with each cycle as a sub-list or something like anumpy vector or a customized class. Clearly if you control the package and how it is used, errors from bad data may not be a concern. But like many Boolean return(s) it is always a problem how to deal with a third possibility. -Original Message- From: Michael F. Stemper To: python-list@python.org Sent: Sat, Apr 23, 2022 8:57 am Subject: Re: Style for docstring On 22/04/2022 21.58, Avi Gross wrote: > Python does have a concept of "truthy" that includes meaning for not just the > standard Booleans but for 0 and non-zero and the empty string and many more > odd things such as an object that defines __bool__ (). > But saying it returns a Boolean True/False valuesounds direct and simple and > informative enough if that is True. > What bothers me is the assumption that anyone knows not so muchjust group > theory but what the argument to the function looks like as a Python object > of some kind. > Does the function accept only some permutation object managed by a specific > module? Will it accept some alternate representation such as a list structure > or other iterator? That's a fair point. However, this function will be the 22nd one in a module for dealing with permutations and groups of permutations. The module has a lengthy docstring explaining the several ways provided to specify a permutation. That way, the same information doesn't need to be written twenty-plus times. > Obviously deeper details would normally be in a manual page or other > documentation but as "permutations" are likely not to be what most people > think about before breakfast, or even after, odd as that may seem, ... I see what you did there :-> -- Michael F. Stemper Psalm 94:3-6 -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
We know some people using "professional" language make things shorteror talk from a point of view different than others and often in otherwise incomprehensible jargon. If a programmer is taking about the algorithm that a function implements, then, yes, they may write "scan" and "return". But if they realize the darn documentation is for PEOPLE asking how to use the darn thing, and want to write in more informal and understandable English, I think it makes more sense to say what the function does as in "scans" and importantly what it "returns" to the user as a result. So if you are taking a programming course and the instructor or textbook is giving imperitave commands, they may well tell youto scan something then calculate something and use a return statement a certain way. I can read many ways and am not particularly bothered by either style but when documenting what is, I prefer proper English (or any otherlanguage) in communicating what it does for them. As with many such things, if you work for a company or with groups of others, it is wise to find out what is expected and do the same as much as reasonable. -Original Message- From: MRAB To: python-list@python.org Sent: Fri, Apr 22, 2022 8:57 pm Subject: Re: Style for docstring On 2022-04-23 00:25, Rob Cliffe via Python-list wrote: > I don't use docstrings much; instead I put a line or two of comments > after the `def ` line. > But my practice in such situations is as per the OP's 3rd suggestion, e.g. > # Returns True if . > I'm curious as to why so many people prefer "Return" to "Returns". > Checking out help() on a few functions in the stdlib, they all used > "Return" or a grammatical equivalent, so this does seem to be a Python > cultural thing. But why? To me, "Returns" begins a description as to > what the function does, whereas "Return" is an imperative. But who is > it addresed to? Is a function considered to be a sentient entity that > can respond to a command? Is it an invocation to the lines of code > following the docstring: "Do this!" Might not the programmer mistakenly > think (if only for a moment) that the imperative is addressed to him? Maybe it's because the function name is often also an imperative, e.g.: >>> import re >>> help(re.search) Help on function search in module re: search(pattern, string, flags=0) Scan through string looking for a match to the pattern, returning a Match object, or None if no match was found. Note "Scan", not "scans". I was going to use 'print' as the example: >>> help(print) Help on built-in function print in module builtins: print(...) print(value, ..., sep=' ', end='\n', file=sys.stdout, flush=False) Prints the values to a stream, or to sys.stdout by default. Optional keyword arguments: file: a file-like object (stream); defaults to the current sys.stdout. sep: string inserted between values, default a space. end: string appended after the last value, default a newline. flush: whether to forcibly flush the stream. but it says "Prints", not "Print"... -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Style for docstring
Python does have a concept of "truthy" that includes meaning for not just the standard Booleans but for 0 and non-zero and the empty string and many more odd things such as an object that defines __bool__ (). But saying it returns a Boolean True/False valuesounds direct and simple and informative enough if that is True. What bothers me is the assumption that anyone knows not so muchjust group theory but what the argument to the function looks like as a Python object of some kind. Does the function accept only some permutation object managed by a specific module? Will it accept some alternate representation such as a list structure or other iterator? Obviously deeper details would normally be in a manual page or other documentation but as "permutations" are likely not to be what most people think about before breakfast, or even after, odd as that may seem, ... And, yes, I know what it means and some users will too. But as all permutations must be even or odd, errors thrown might be based on whether the data structure has valid contents or is so complex that it uses up all system resources, I would think. So the docstring could be fairly short and something like: Given a permutation in Returns the Boolean value True if it is graded as or False if or an exception if the argument is not valid. As noted by others, Many things can be returned including multiple values where perhaps the second one tells if there was an error but thatthe user can ignore or not even catch. -Original Message- From: Chris Angelico To: python-list@python.org Sent: Fri, Apr 22, 2022 6:33 pm Subject: Re: Style for docstring On Sat, 23 Apr 2022 at 08:24, <2qdxy4rzwzuui...@potatochowder.com> wrote: > > On 2022-04-22 at 15:35:15 -0500, > "Michael F. Stemper" wrote: > > > On 22/04/2022 14.59, Chris Angelico wrote: > > > On Sat, 23 Apr 2022 at 05:56, Michael F. Stemper > > > wrote: > > > > > > > > I'm writing a function that is nearly self-documenting by its name, > > > > but still want to give it a docstring. Which of these would be > > > > best from a stylistic point of view: > > > > > > > > > > > > Tells caller whether or not a permutation is even. > > > > > > > > Determines if a permutation is even. (Alternative is that it's odd.) > > > > > > > > Returns True if permutation is even, False if it is odd. > > > > > > > > > > I'd go with the third one, but "Return" rather than "Returns". Or > > > possibly "Test whether a permutation is even". > > > > "So let it be written. So let it be done." > > "Test whether a permutation is even," while technically factual, leaves > the reader to wonder what form the result takes, and what happens to > that result. While it's definitely possible to have other results and other ways to deliver them, the return of a boolean would be the most obvious default. > Do you want callers of the function also to assume that True means that > the permutation is even? There are other reasonable strategies, such as > an enumerated type (whose items are Even, Odd, and FileNotFound), or > throwing an exception if the permutation is odd. I'm assuming that the function is called something like "is_even()" and that it either is a method on a permutation object, or its parameters make it very clear what the permutation is. If it returns an enumeration, I would say that in the docstring. If the docstring doesn't say, I would assume it returns True or False. > I prefer the "return" (rather than "returns") version of the third > option. Assuming that the programmers are familiar with the domain, the > other two leave out important information. Core Python methods and functions seem to prefer either "Return ..." or "Verb the thing" where the result is implicit (eg str.zfill.__doc__ which says "Pad a numeric string..."). Both are used extensively. Neither form leaves out anything that wouldn't be the obvious default. We don't need to say "Figures out algorithmically whether the permutation is even. If it is, will return True; if it isn't, will return False; if something goes wrong, will raise an exception". This is Python; we know that if something goes wrong, an exception is raised. (Though it can help to say WHICH exception will be raised under WHAT circumstances). Some things are obvious. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Why no list as dict key?
This does raise an issue, Chris, if you use the method of making a tuple companion for a list at a specific time just for use as a dictionary key, then later change the list, you can end up with various situations. Obviously the changed list can not only not access the stored item, but if converted again to a tuple, may address a different item. I can see many scenarios with abandoned dictionary items that are never deleted and can only be reached by examining all items in the dictionary. So mutability is only one concern. If you actually mutate your data, ... I am thinking as an example about a program I wrote ages ago that deals with equations in symbolic form and maintains a collection of forms of the equation it is trying to take a derivative or integral of by applying an assortment of typographic rules. I mean commutative lawand others. You do not want to keep adding the same item into the data structure (such as a queue) repeatedly. So something like a dictionary (or set) can be a good way to store unique items. But the items are some complex lists so the above discussion qualifies. Of course the tuple conversion for a nested structure would need to have made a deep copy. The question in the above is how to make sure that taking a next attempt off the queue deals with the dictionary of tried items. In this case, unless you use it to find a solution, keeping it in the dictionary to avoid repeating, makes sense. And another thought is that mapping a list to a tuple has another possible drawback. What if I have both a list and tuple with the same structure which I want as keys? I can imagine then converting the list in some imaginative ways. For example, embed the list in another list whose first item is "from-tuple" or something. Life is complicated. Then you die. -Original Message- From: Chris Angelico To: python-list@python.org Sent: Wed, Apr 20, 2022 3:49 pm Subject: Re: Why no list as dict key? On Thu, 21 Apr 2022 at 05:30, Sam Ezeh wrote: > > Repeating the above points, here is an example of what would happen if > you tried. Dictionaries require their keys to be immutable as > under-the-hood they use hash tables and they'd fail when the > underlying values are allowed to change. > > ``` > >>> class HashableList(list): > ... def __hash__(self): > ... return functools.reduce(operator.xor, [key * value for > key, value in enumerate(self)], 5) Quickie: I'd be inclined to define hash on top of a tuple's hash, rather than try to design my own and hope that it's suitable. "return hash(tuple(self))" is a good demonstration of the parallel. Otherwise, good demonstration of the problem. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: code confusion
As usual, without very clear and precise instructions and parameters, the
answers may not quite fit.
It looks like you are asked two and only two questions.
The first is asking how many numbers you want.
Before continuing, you need to make sure that is a valid number as many answer
will throw an exception.
The next line, complex as it is, asks for one long answer and does not check
anything and breaks rapidly unless you use just normal integer representation.
Yes, it ignores any entry after the "i"th but if you want valid entries, you
might want to evaluate them in a loop perhaps one at a time and keep going till
you have 'i" valid ones.
I do suggest you not use the variable name of "i" for many reasons as modern
languages allow more meaningful names like, well, "n"!
I understand using i, j, k in some nested loops but here I would haveused
something like howMany and verified the number was an integer larger than 0.
As for getting the second largest number, there is nothing wrong with
determining it the hard way. Of course for some people, it is more intuitive to
sort the uniqued data and simply choose the 2nd entry from the end. Some python
modules allow you tosee the ranks of various entries and you can simply choose
the one of second rank.
But if this is HW, you are being asked to do things the old-fashioned way! LOL!
-Original Message-
From: Dennis Lee Bieber
To: python-list@python.org
Sent: Fri, Apr 15, 2022 2:31 pm
Subject: Re: code confusion
On Fri, 15 Apr 2022 08:41:20 +0100, Tola Oj
declaimed the following:
>i = int(input())
Obtain a single /integer/ from stdin -- note: any extraneous characters
on the input line will result in a failure to convert from textual
representation to internal/binary integer
>lis = list(map(int,input().strip().split()))[:i]
Obtain a line from stdin containing space separated /integer/
representations. Split the line at the spaces. Convert each "word" to
internal/binary integer. Keep up to at most "i" integers. Note that the
position of the [:i] could be at
... .split()[:i]
The difference being that the provided code is converting all "words" on
the input into integers and then keeping the first "i"; putting the [:i]
after .split() means only the first "i" words are kept, and hence only that
many need to be converted to integer.
>z = max(lis)
Determine largest value in the list of integers
>while max(lis) == z:
>lis.remove(max(lis))
WHILE the largest value in the (current) list matches the initially
determined maximum value... remove that value from the list.
Rather than repeat "max(lis)" in the .remove() invocation, just pass it
"z" (the WHILE has already confirmed that the maximum "z" is found in the
list, so why recompute the maximum).
Note: Python indentation is significant -- the above .remove() line
needs to be indented. Presuming your code was properly indented please find
a posting client that doesn't reformat leading indentation.
>
>print (max(lis))
>
Display the new list maximum value after removing all instances of the
initial maximum value.
>this is an answer to a question from the discussion chat in hackerrank. i
>didn't know the answer so i found an answer that fitted well to the
>question, however i struggle to understand the use of some of the methods
>and functions the person has used. my major questions are: 1. what does
>"[:i]" mean
Learn the contents of the Library Reference Manual -- you don't need to
memorize it all, but should at least know the major groupings...
https://docs.python.org/3/library/stdtypes.html#common-sequence-operations
> 2. is there
>another i could write this code using if statement?
There are many ways to rewrite that...
UNTESTED
i = int(input("How many integers are to be considered?"))
ls = [int(wd) for wd in input(
"enter space separated integers"
).split()[:i]]
maximum = max(ls)
while maximum in ls:
ls.remove(maximum)
print(ls)
Remove the first line, and the [:i], and the code will happily process
for however many integers are provided on the input line.
The while/remove loop can be replaced with
ls = [itm for itm in ls if itm != maximum]
which only requires one pass through the list; while/remove has to scan the
list to see if maximum is in it, then has to scan it a second time to for
the .remove() to find where in the list it is found.
Or...
for _ in range(ls.count(maximum)):
ls.remove(maximum)
where _ is a "junk/temp" value that we don't care about -- we only want to
loop once for EACH maximum value
Or...
while maximum in ls:
del ls[ls.index(maximum)]
--
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: dict.get_deep()
Kirill,
There are many related needs and issues and solutions such as how to parse XML
content and do all kinds of tree searches for "patterns" that multiple modules
have been created to deal with.
My impression here was of a simpler request to allow a list of keys to be
applied in sequence. The example wanted the list to be successive arguments in
a call to a method, but obvious variants would be for a single argument
containing atuple or list or any interator that would probe a tree of
possibilities while anchored to the top.
This is a much easier task with many solutions offered.
-Original Message-
From: Kirill Ratkin via Python-list
To: python-list@python.org
Sent: Mon, Apr 4, 2022 3:40 am
Subject: Re: dict.get_deep()
Hello,
Yes, I misunderstood as well because started to think about pattern
matching which is good but this is not subject the question was about.
Sorry for my mistake.
Because question was about 'builtin' function which means stdlib
function implemented in python itself or even in C.
It seems, maybe I miss again, but we are talking about similar ideas
behind 'xpath' or 'jsonpath' or even 'LINQ'. We want to find some 'dsl'
which give us simple and safe way to get deeply nested values from dict.
There are several similar solutions on pypi
(https://pypi.org/project/dpath/, https://pypi.org/project/path-dict/).
But maybe (and maybe I miss again) we talk about language embedded
solution like operator ? or ??.
For example deep dict extraction could look like: street =
data["users"]?[0]?["address"]?["street"]?.
// BR
04.04.2022 2:24, Avi Gross via Python-list пишет:
> I may have misunderstood something.
> The original post in this subject sounded to ME likethey had nested
> dictionaries and wanted to be ableto ask a method in the first dictionary
> totake an unspecified number of arguments thatwould be successive keys and
> return the results.
> I mean if A was a dictionary containing saycities and it had an alphabetical
> index of lettersA to Z and those contained dictionaries ofsay last names as
> additional dictionaries andso on, then they wanted to perhaps say;
> A.getdeep("Boston", "S", "Smith", "Address", default="None")
> But the replies I am seeing look so different that I mayhave missed something
> as it seems more about usingpattern matching on the data used to make the
> dictionariesor something.
> So I was happy to see Marco suggesting a function alongthe lines of my
> thought process. But I have another thought.A stand-alone function along his
> lines might be fine. Buta method built into a general Dictionary class is
> anotherthing as it asks a method in one dictionary to march aroundinto other
> dictionaries. So I wonder if a better methodis sort of recursive.
> If you had a class like dictionary that had a getdeep function,and it got
> called with N arguments, and perhaps a namedargument supplying a default,
> then would it make sensefor the function checking to see if the FIRST
> argument canbe found as a key to the current dictionary.
> If arguments remain then it should expect to finda result that is a
> dictionary (or perhaps some otherobject that supports the getdeep() protocol
> and ask thatobject to operate on the N-1 remaining arguments, passingthe
> default along too.
> If the request is valid, after some iterations an object willhave a method
> invoked with a single argument (plus default)and a value passed back up the
> chain. For any errors alongthe way, the default would be returned.
> Is this closer to the spirit of the request? I view this versionof nested
> dictionaries as a sort of tree structure with variablebranches along the way.
> So an approach like this could makesense and perhaps Python could be updated
> eventually to havesome objects support such a protocol.
> Of course you could sort of do it yourself by subclassing somethingand making
> changes but that may not work for what is already asort of built-in data
> structure but could work for one of many variantsalready implemented in
> modules.
>
>
>
> -Original Message-
> From: Marco Sulla
> To: Peter J. Holzer
> Cc: python-list@python.org
> Sent: Sun, Apr 3, 2022 5:17 pm
> Subject: Re: dict.get_deep()
>
> On Sun, 3 Apr 2022 at 21:46, Peter J. Holzer wrote:
>>>> data.get_deep("users", 0, "address", "street", default="second star")
>> Yep. Did that, too. Plus pass the final result through a function before
>> returning it.
> I didn't understand. Have you added a func parameter?
>
>> I'm not sure whether I considered this when I wrote it, but a function
>> has the advantage of working with every class which c
Re: dict.get_deep()
I may have misunderstood something.
The original post in this subject sounded to ME likethey had nested
dictionaries and wanted to be ableto ask a method in the first dictionary
totake an unspecified number of arguments thatwould be successive keys and
return the results.
I mean if A was a dictionary containing saycities and it had an alphabetical
index of lettersA to Z and those contained dictionaries ofsay last names as
additional dictionaries andso on, then they wanted to perhaps say;
A.getdeep("Boston", "S", "Smith", "Address", default="None")
But the replies I am seeing look so different that I mayhave missed something
as it seems more about usingpattern matching on the data used to make the
dictionariesor something.
So I was happy to see Marco suggesting a function alongthe lines of my thought
process. But I have another thought.A stand-alone function along his lines
might be fine. Buta method built into a general Dictionary class is
anotherthing as it asks a method in one dictionary to march aroundinto other
dictionaries. So I wonder if a better methodis sort of recursive.
If you had a class like dictionary that had a getdeep function,and it got
called with N arguments, and perhaps a namedargument supplying a default, then
would it make sensefor the function checking to see if the FIRST argument canbe
found as a key to the current dictionary.
If arguments remain then it should expect to finda result that is a dictionary
(or perhaps some otherobject that supports the getdeep() protocol and ask
thatobject to operate on the N-1 remaining arguments, passingthe default along
too.
If the request is valid, after some iterations an object willhave a method
invoked with a single argument (plus default)and a value passed back up the
chain. For any errors alongthe way, the default would be returned.
Is this closer to the spirit of the request? I view this versionof nested
dictionaries as a sort of tree structure with variablebranches along the way.
So an approach like this could makesense and perhaps Python could be updated
eventually to havesome objects support such a protocol.
Of course you could sort of do it yourself by subclassing somethingand making
changes but that may not work for what is already asort of built-in data
structure but could work for one of many variantsalready implemented in modules.
-Original Message-
From: Marco Sulla
To: Peter J. Holzer
Cc: python-list@python.org
Sent: Sun, Apr 3, 2022 5:17 pm
Subject: Re: dict.get_deep()
On Sun, 3 Apr 2022 at 21:46, Peter J. Holzer wrote:
>
> > > data.get_deep("users", 0, "address", "street", default="second star")
>
> Yep. Did that, too. Plus pass the final result through a function before
> returning it.
I didn't understand. Have you added a func parameter?
> I'm not sure whether I considered this when I wrote it, but a function
> has the advantage of working with every class which can be indexed. A
> method must be implemented on any class (so at least dict and list to be
> useful).
You're right, but where to put it? I don't know if an iterableutil package
exists. If included in the stdlib, I don't know where to put it. In
collections maybe?
PS: if you're interested, here is my implementation:
def get_deep(self, *args, default=_sentinel):
r"""
Get a nested element of the dictionary.
The method accepts multiple arguments or a single one. If a single
argument is passed, it must be an iterable. This represents the
keys or indexes of the nested element.
The method first tries to get the value v1 of the dict using the
first key. If it finds v1 and there's no other key, v1 is
returned. Otherwise, the method tries to retrieve the value from v1
associated with the second key/index, and so on.
If in any point, for any reason, the value can't be retrieved, the
`default` parameter is returned if specified. Otherwise, a
KeyError or an IndexError is raised.
"""
if len(args) == 1:
single = True
it_tpm = args[0]
try:
len(it_tpm)
it = it_tpm
except Exception:
# maybe it's a generator
try:
it = tuple(it_tpm)
except Exception:
err = (
f"`{self.get_deep.__name__}` called with a single " +
"argument supports only iterables"
)
raise TypeError(err) from None
else:
it = args
single = False
if not it:
if single:
raise ValueError(
f"`{self.get_deep.__name__}` argument is empty"
)
else:
raise TypeError(
f"`{self.get_deep.__name__}` expects at least one argument"
)
obj = self
for k in it:
try:
obj = obj[k]
except (KeyError, IndexError) as e:
if default is _sentinel:
raise e from None
Re: How to detect an undefined method?
The question seems to be how or whether you can check Python code in
advance for any instances of a method for an object being called that is
not instantiated. Right?
As Kirill points out, Python can be quite dynamic. I can think of oodles
of ways checking would not work well in a static examination of the code.
Just to mention a few, I can have a collection of objects (such as a list)
and I may iterate on it to take each object and call a specific method.
Some objects may have the method and others may not. The dynamic
code may easily include objects that conform to expectations or not.
For example, you ay be reading in saved objects from a file or objects
are being created as the user interacts.
And is it an error if a program is written to detect and perhaps correct
errors? What if I want to display an object and in a "try" block I ask to
run one of several methods and on failure, try the next. Some objects
may have a method for full_profile() while others just might have a name()
and yet others might have no known way and will be shown as ANONYMOUS.
There also seem to be ways to extend an existing object or the entire class
after
it has been created. Objects with multiple inheritance also may be headache.
So I suspect warnings for some cases make sense but a tool that catches
everything,
maybe not easy or even possible.
You can, of course, make your code a tad more bulletproof, there are ways you
can
have your code deliberately check if the object has that method before trying to
invoke it, or you can handle exceptions generated if it doesn't.
-Original Message-
From: Kirill Ratkin via Python-list
To: python-list@python.org
Sent: Sun, Mar 27, 2022 2:29 pm
Subject: Re: How to detect an undefined method?
I just started to think from your example with method 'err' of logger
object.
In this particular case you can check method 'err' exists or not before
call this.
But if you mean general case ... . If for example I use some library
which uses another library and someone just 'typo' there ...
Here is example from my code:
calc: Calculator = Calculator()
...
actions: Dict[str, Callable] = {
"s.sysinfo": calc.get_system_info,
"s.setloglevel": calc.set_log_level,
...
}
...
def do_action(action: str, params: Dict[str, str]) -> ActionResult:
return await actions[action](params)
And if I make mistake and type 'calc.get_s*i*stem_info' instead
'calc.get_s*y*stem_info. Error appears on rutime stage only.
And neither 'mypy --strict' or 'pyre' can find such error.
I guess there is not warranty to detect such sitations in huge codebase.
It's python dynamic nature.
May be dynamic checkers can help in such situations ...
27.03.2022 20:07, Manfred Lotz пишет:
> On 3/27/22 18:57, Kirill Ratkin wrote:
>> Hi
>>
>> You can get all methods of your object and check the method you want to call
>> is
>> there or not.
>>
>> |methods = [method for method in dir() if
>> callable(getattr(, method))] if 'method_you_need' in methods:
>> . // BR |
>>
> I don't understand how this may help. Assume somebody has a codebase of 15T
> lines of Python code. How do you want to apply your method?
>
> But perhaps I overlook things.
>
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
