Re: pip/setuptools: Entry points not visible from pkexec-root-environment
On Mon, 19 Dec 2022 at 19:38, wrote: > > Dear Chris, > thanks for asking back and my apologize for being to broad in my way of > asking (in a foreign language). > > Am 19.12.2022 07:40 schrieb Chris Angelico: > > Hmm, then I'm not sure what you're *losing* here. The problem, as I > > understand it, is that the scripts are getting installed into > > /usr/local/bin (which is on PATH at the time they're installed), but > > pkexec has a restricted PATH. So if you use which before pkexec'ing, > > wouldn't you find the scripts, and then be able to run them without > > regard to PATH? > > Absolut correct. This works. > > The question is if this is a "good" or "elegant" way from the viewpoint > of different communities/projects (e.g. Python, setuptools, pip, other > build-systems, distros, something I forgot). > Ah! Then, my response is: Yes, it is definitely a good/elegant solution. I've often created scripts that install into somewhere (eg systemd, cron, ifupdown) by using 'which' to figure something out, and then posting the full path into the corresponding file. It's a convenient way to ensure that the exact same program would be used. Notably, "which python3" will ensure that you run on the currently-active Python interpreter; this includes virtual environments. Very convenient. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/setuptools: Entry points not visible from pkexec-root-environment
On Mon, 19 Dec 2022 at 17:30, wrote: > > Am 18.12.2022 22:37 schrieb Mats Wichmann: > > the which command uses your PATH, so I'm not sure you're buying > > anything new there > > I'm using which before entering pkexec. ;) > > I'll show a demonstrator project later. Hmm, then I'm not sure what you're *losing* here. The problem, as I understand it, is that the scripts are getting installed into /usr/local/bin (which is on PATH at the time they're installed), but pkexec has a restricted PATH. So if you use which before pkexec'ing, wouldn't you find the scripts, and then be able to run them without regard to PATH? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: String to Float, without introducing errors
On Mon, 19 Dec 2022 at 07:57, Stefan Ram wrote: > G = Decimal( 6.6743015E-11 ) > r = Decimal( 6.371E6 ) > M = Decimal( 5.9722E24 ) What's the point of using Decimal if you start with nothing more than float accuracy? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/setuptools: Entry points not visible from pkexec-root-environment
On Mon, 19 Dec 2022 at 04:56, wrote: > > Hello, > > when I install a package on a GNU/Linux system via "sudo python3 -m pip > install -e ." that defines entry points in its pyproject.toml the entry > point starter scripts are located in /usr/locale/bin. > > That folder is in PATH for "regular" root users and by "sudo su" roots > users. > > But I need to start that entry points via "pkexec". > But in the environment started by "pkexec" the PATH does not contain > /usr/local/bin. > > So what can I do? > > I don't need a hack or workaround but an "elegant" solution. Does it have to be in path? Can't you say /usr/local/bin/entrypointname? Not sure what constitutes an elegant solution here. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Installation hell
On Mon, 19 Dec 2022 at 06:10, Mats Wichmann wrote: > Why? Python is a command-line tool to process a language, Similar to > many other languages - Go, for example. Or a C/C++ compiler. *Or* you > can choose to use someone's wrapping of that process inside an > Integrated Development Environment. There are tons that support Python > and let you run your code from within the editor environment without > having to go open a cmd.exe or powershell box. Most of those are > external, but the comes-with-Python IDLE works well, too. I wouldn't bother responding to these sorts of people. They have already decided that it's impossible to find any sort of decent IDE for Python (despite pretty much every editor out there having Python support), are deathly afraid of command lines, and yet feel the need to join a mailing list to tell us all that. You won't convince them of anything. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Single line if statement with a continue
On Sun, 18 Dec 2022 at 10:59, wrote: > > If a compiler or interpreter HAPPILY (as happy as machines/code get) compiles > or interprets your code without errors every time you use it a certain way, > then it is not wrong to use it. Of course if it subject to change or already > deprecated, ... > Source code is, first and foremost, for programmers to read. You're confusing it with binary executables. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: String to Float, without introducing errors
On Sun, 18 Dec 2022 at 09:46, Stefan Ram wrote: > > Grant Edwards writes: > >Yes, fixed point (or decimal) is a better fit for what he's doing. but > >I suspect that floating point would be a better fit for the problem > >he's trying to solve. > > I'd like to predict that within the next ten posts in this > thread someone will mention "What Every Computer Scientist > Should Know About Floating-Point Arithmetic". > > |>>> 0.1 + 0.2 - 0.3 > |5.551115123125783e-17 > Looks like someone just did. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: String to Float, without introducing errors
On Sun, 18 Dec 2022 at 08:22, Grant Edwards wrote: > > On 2022-12-17, Chris Angelico wrote: > > >> It was the rounding rounding error that I needed to avoid (as Peter > >> J. Holzer suggested). The use of decimal solved it and just in > >> time. I was about to truncate the number, get each of the > >> characters from the string mantissa, and then do something like > >> this: > >> > >> 64550.727 > >> > >> 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001) > >> > >> Now I do not need to! > > > > It sounds like fixed-point arithmetic might be a better fit for what > > you're doing. > > Yes, fixed point (or decimal) is a better fit for what he's doing. but > I suspect that floating point would be a better fit for the problem > he's trying to solve. > Hard to judge, given how little info we have on the actual problem. Could go either way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: String to Float, without introducing errors
On Sun, 18 Dec 2022 at 07:46, Paul St George wrote: > > Thanks to all! > It was the rounding rounding error that I needed to avoid (as Peter J. Holzer > suggested). The use of decimal solved it and just in time. I was about to > truncate the number, get each of the characters from the string mantissa, and > then do something like this: > > 64550.727 > > 64550 + (7 * 0.1) + (2 * 0.01) + (7 * 0.001) > > Now I do not need to! It sounds like fixed-point arithmetic might be a better fit for what you're doing. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Single line if statement with a continue
On Thu, 15 Dec 2022 at 16:29, Thomas Passin wrote: > > PEP-8, which is Guido's style guide and generally good to follow, does > not completely discourage single-line usage like the example. It's not > clear to me how Chris's example fits into the guidelines. > > PEP-8: > "While sometimes it’s okay to put an if/for/while with a small body on > the same line, never do this for multi-clause statements. > ... > # Wrong: > if foo == 'blah': do_blah_thing() > for x in lst: total += x > while t < 10: t = delay() > " > > If the one-liner were not in a multi-statement block, it would be all > right with PEP-8. Not sure what your point is about it being "in" a multi-statement block - PEP 8 has nothing to say about that. What it's saying is that you shouldn't do this: if foo == 'blah': one(); two(); three() And I agree; if you're putting more than one statement after your 'if', it's generally clearest to have it on multiple lines. But a simple "continue" or "break" statement works just fine on the same line as the if. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Single line if statement with a continue
On Thu, 15 Dec 2022 at 14:41, Aaron P wrote: > > I occasionally run across something like: > > for idx, thing in enumerate(things): > if idx == 103: > continue > do_something_with(thing) > > It seems more succinct and cleaner to use: > > if idx == 103: continue. > > Of course this would be considered an anti-pattern, and Flake8 will complain. > > Any opinions, or feedback on the matter. Nothing at all wrong with writing that on a single line. If you have issues with Flake8 not accepting your choices, reconfigure Flake8 :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 double quote behavior
On Wed, 14 Dec 2022 at 08:19, Roel Schroeven wrote: > > Chris Angelico schreef op 13/12/2022 om 20:01: > > On Wed, 14 Dec 2022 at 06:00, Roel Schroeven wrote: > > > > > > Stefan Ram schreef op 13/12/2022 om 8:42: > > > > "John K. Parejko" writes: > > > > >I was just burned by this, where some tests I’d written > > > > >against an sqlite database did not fail in the way that they > > > > >“should” have, because of this double-quoted string issue. > > > > > > > >In standard SQL, double quotes denote identifiers that are > > > >allowed to contain special characters. > > > Or that are equal SQL keywords, which can be a reason to double-quote > > > them. SQL engines sometimes add new keywords; explicitly marking string > > > literals as string literals prevents future conflicts and confusion. > > > > > > Perhaps it's a better idea to use [identifier] or `identifier` instead > > > though (I just learned about those on > > > https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is > > > used in MS Access and SQL Server, `` is used in MySQL) but both work in > > > SQLite. That should prevent any ambiguity and confusion, if it doesn't > > > bother you too much that it's not standard SQL. > > > > > > > Why not just use "identifier" which is standard SQL? > > If you accidentally type [identifire] or `identifire`, SQLite will > produce an unknown identifier error, alerting you immediately to your typo. > If you accidentally type "identifire", SQLite will silently treat it as > a string literal instead of an identifier, causing more difficult to > diagnose problems. > Okay, so. exactly the same as if you use standard double quotes, but change the configuration option. So the options are: make everything worse for everyone by exacerbating the problem of non-standard identifier quoting, or get this API so SQLite can be configured, like the OP actually asked for. Yeah. Let's not do the wrong thing. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 double quote behavior
On Wed, 14 Dec 2022 at 06:00, Roel Schroeven wrote: > > Stefan Ram schreef op 13/12/2022 om 8:42: > > "John K. Parejko" writes: > > >I was just burned by this, where some tests I’d written > > >against an sqlite database did not fail in the way that they > > >“should” have, because of this double-quoted string issue. > > > >In standard SQL, double quotes denote identifiers that are > >allowed to contain special characters. > Or that are equal SQL keywords, which can be a reason to double-quote > them. SQL engines sometimes add new keywords; explicitly marking string > literals as string literals prevents future conflicts and confusion. > > Perhaps it's a better idea to use [identifier] or `identifier` instead > though (I just learned about those on > https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is > used in MS Access and SQL Server, `` is used in MySQL) but both work in > SQLite. That should prevent any ambiguity and confusion, if it doesn't > bother you too much that it's not standard SQL. > Why not just use "identifier" which is standard SQL? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Top level of a recursive function
On Wed, 14 Dec 2022 at 05:01, Mats Wichmann wrote: > > On 12/13/22 10:36, Chris Angelico wrote: > > On Wed, 14 Dec 2022 at 03:35, Michael F. Stemper > > wrote: > >> > >> It's easy enough -- in fact necessary -- to handle the bottom > >> level of a function differently than the levels above it. What > >> about the case where you want to handle something differently > >> in the top level than in lower levels? Is there any way to tell > >> from within a function that it wasn't invoked by itself? > >> > > > > Why does it have to be the same function? > > > > def _sort_recursive(stuff, keys, start, end): > > """imagine a nice implementation of some sorting algorithm here""" > > > > def sort(stuff, key=None): > > if key: > > keys = [key(x) for x in stuff] > > else: > > keys = stuff > > return _sort_recursive(stuff, 0, len(stuff)) > > if some support for this position is needed, this is roughly how the > stdlib glob() function is implemented. > Yeah, lots of things are done that way. You'll find a variety of naming conventions around different languages and libraries, including "_low_FUNCTIONNAME" or "_internal_FUNCTIONNAME" etc. It's definitely easier than trying to mess with tracking toplevel status. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Top level of a recursive function
On Wed, 14 Dec 2022 at 03:35, Michael F. Stemper wrote: > > It's easy enough -- in fact necessary -- to handle the bottom > level of a function differently than the levels above it. What > about the case where you want to handle something differently > in the top level than in lower levels? Is there any way to tell > from within a function that it wasn't invoked by itself? > Why does it have to be the same function? def _sort_recursive(stuff, keys, start, end): """imagine a nice implementation of some sorting algorithm here""" def sort(stuff, key=None): if key: keys = [key(x) for x in stuff] else: keys = stuff return _sort_recursive(stuff, 0, len(stuff)) With purely recursive functions (where every call to the function truly could have been a top-level call - a lot of mathematical functions work out this way), it makes sense to call the externally-callable function recursively; but for anything more messy, it's usually easiest to make the recursive part internal, and then have a top-level function that calls into the recursive one. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 double quote behavior
On Wed, 14 Dec 2022 at 00:30, Thomas Passin wrote:
>
> On 12/13/2022 4:09 AM, Chris Angelico wrote:
> > On Tue, 13 Dec 2022 at 19:52, Roel Schroeven wrote:
> >> Like Lars Liedtke this is not an exact answer to your question, but you
> >> can side-step the issue by using parametrized queries, i.e. instead of
> >>
> >> cur.execute('SELECT name, location FROM persons WHERE name = "John
> >> Doe"')
> >>
> >> do
> >>
> >> cur.execute('SELECT name, location FROM persons WHERE name = ?',
> >> ('John Doe',))
> >>
> >
> > That's the wrong behaviour though. According to the SQL standard, the
> > second query should be equivalent to this:
> >
> > cur.execute("SELECT name, location FROM persons WHERE name = 'John Doe'")
> >
> > What the OP wanted was like your first query, and proper DBMSes like
> > PostgreSQL will handle it accordingly. The question is how to get
> > SQLite3 to also do so.
>
> From reading the SQLite3 documentation on this issue (not from personal
> experience), in fact the second form is actually what one wants, even if
> SQLite3 will usually handle the first form correctly.
No, the two have distinct semantics. BOTH are valid, they just mean
different things.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: sqlite3 double quote behavior
On Tue, 13 Dec 2022 at 19:52, Roel Schroeven wrote:
> Like Lars Liedtke this is not an exact answer to your question, but you
> can side-step the issue by using parametrized queries, i.e. instead of
>
> cur.execute('SELECT name, location FROM persons WHERE name = "John
> Doe"')
>
> do
>
> cur.execute('SELECT name, location FROM persons WHERE name = ?',
> ('John Doe',))
>
That's the wrong behaviour though. According to the SQL standard, the
second query should be equivalent to this:
cur.execute("SELECT name, location FROM persons WHERE name = 'John Doe'")
What the OP wanted was like your first query, and proper DBMSes like
PostgreSQL will handle it accordingly. The question is how to get
SQLite3 to also do so.
I don't use SQLite3 much so I'm not really one to judge, but maybe it
would be worth exposing the sqlite3_db_config() function to Python?
Yes, it would be more than a trivial change, but it should be
reasonably straight-forward. In order to be useful, it would probably
also need an associated IntEnum for all those lovely opaque numbers
that define the verbs.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Does one have to use curses to read single characters from keyboard?
On Mon, 12 Dec 2022 at 09:24, Barry Scott wrote: > You would need to have a loop that collected all the utf-8 bytes of a single > code point. > You can to look at the first byte of know if the utf-8 is 1, 2, 3 or 4 bytes > for a code point. And cope with escape sequences too - if you press an arrow key, for instance, you'll get a multi-character string to signal that. This is why it's probably easier to let someone else do the work. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: FTP without username and password
On Wed, 7 Dec 2022 at 02:51, ^Bart wrote: > > Hi Guys, > > usually I use this code on my Debian Bullseye: > > # python3 -m pyftpdlib -i 192.168.0.71 -p 21 -d /home/my_user/ftp > > It works, it's simply easy and perfect but... a device in my lan needs a > ftp folder without username and password! > > I tried to search on internet how to set the code above to be available > without username and password but... I didn't understand how to fix it :\ > > Obviously I could use a workaround like Samba or another machine where I > have a Vsftp server but... I'd like to fix Python! ;) > I assume it HAS to be FTP, otherwise you'd set up something much more secure like ssh (or scp or sshfs, which are built on it), done with an authorized key rather than a password. In general, "anonymous FTP" is done technically with a username and password. Can you look at how the device tries to connect, and then make that username (probably "anonymous") and that password (could be anything, traditionally was an email address) valid for fetching? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: argparse — adding a --version flag in the face of positional args
On Tue, 29 Nov 2022 at 12:37, Loris Bennett wrote: > > Mats Wichmann writes: > > > On 11/27/22 16:40, Skip Montanaro wrote: > >> I have a script to which I'd like to add a --version flag. It should print > >> the version number then exit, much in the same way --help prints the help > >> text then exits. I haven't been able to figure that out. I always get a > >> complaint about the required positional argument. > >> I think I could use something like nargs='*', but that would push > >> off > >> detection of the presence of the positional arg to the application. > >> Shouldn't I be able to tell argparse I'm going to process --verbose, then > >> exit? > > > > ummm, hate to say this, but have you checked the documentation? this > > case is supported using an action named 'version' without doing very > > much. > > I hadn't noticed the action 'version'. I just use > > parser.add_argument( > "-v", "--version", action="store_true", dest="version", > help="print version" > ) > That's still going to validate the rest of the args though - notably, you can't omit any mandatory arguments (like a subcommand). The version and help actions are implemented pretty simply, actually. They're just little action classes that do their action immediately on getting triggered. It should be easy enough to make any action you want that way. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: In code, list.clear doesn't throw error - it's just ignored
On Thu, 24 Nov 2022 at 06:26, Stefan Ram wrote: > > Jon Ribbens writes: > >If you want to catch this sort of mistake automatically then you need > >a linter such as pylint: > > output > > , line 1 > list.clear > Warning: Attribute used as statement. > > , line 5 > list.clear > Warning: Attribute used as statement. > > source code > > import ast, sys > > def check( point, source ): > if isinstance( point, ast.Expr ) and\ > type( point.value )== ast.Attribute: > print( ", line", point.lineno, file=sys.stderr ) > print( source.split( '\n' )[ point.lineno-1 ], file=sys.stderr ) > print\ > ( "Warning:", "Attribute used as statement.", file=sys.stderr ) > print() > > def mylinter( source ): > for point in ast.walk( ast.parse( example )): >check( point, source ) > > example = """\ > list.clear > list.clear() > x = list.clear > print( list.clear ) > list.clear > """ > > mylinter( example ) > Uhh, yes? You just created an extremely simplistic linter. Your point? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Passing information between modules
On Mon, 21 Nov 2022 at 16:26, dn wrote: > Am put-off by the 'smell' of subverting/adapting names like print() = > surprise/confusion factor - but I think I understand where you're going. To be fair, redefining the "print" function IS one of the reasons that it's no longer a statement. Though I would generally recommend maintaining its signature and purpose. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Passing information between modules
On Mon, 21 Nov 2022 at 09:37, Dan Kolis wrote: > > Using sys.stdout / is simply nonsense. The more I think about it, the more I > realise how bad it is. > > Going on about it endlessly seems pointless. > > If the even mini threading thing is turned on, now what ? some other module > eats the message intended for a different module ? A state machine with its > own matching code in sends and receives to reuse the unwanted singular value ? > > The precise rules for establishing a variable without the global keyword is > not obvious, or catcat of anything by leaving a empty set dangling initially. > > *especially* if the program is fundamentally open ended, that is, there could > be considerable new functions sharing ideas all over, planning ahead for as > good 45 seconds is a lot better then endless hacking any old way. You should print this out and get it on a t-shirt. It's a perfect example of AI-generated text. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Logging
On Sun, 20 Nov 2022 at 12:27, Cameron Simpson wrote: > > On 19Nov2022 18:26, Thomas Passin wrote: > >>The alternative is to just replace every calling function which uses > >>my_ugly_debug() to directly call a logging.whatever() call. > > > >Maybe a place for a decorator... > > Indeed, though I don't think the OP is up to decorators yet. > > But really, is there any problem which cannot be solved with a > decorator? I've even got a @decorator decorator for my decorators. > Do you have a @toomanydecorators decorator to reduce the number of decorators you need to decorate your decorators? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: In code, list.clear doesn't throw error - it's just ignored
On Wed, 16 Nov 2022 at 10:11, wrote: > > That is clear, Cameron, but on my python interpreter values evaluated on the > command line ARE saved: > > >>> numb = 5 > >>> 5 + numb > 10 > >>> numb > 5 > >>> _ + _ + 1 > 11 That's a REPL feature. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Are these good ideas?
On Tue, 15 Nov 2022 at 09:38, Barry wrote: > > > > > On 14 Nov 2022, at 22:06, Thomas Passin wrote: > > > > For parameter passing like your #2, I have packaged them into a dictionary > > and passed that around. It was easy enough, and worked well. > > > I used to use a dict but having been burnt with issues now create a class. > > With a class you can add accessor methods that allow you to run debug code as > values are changed. Also you can check that values being set are spelt > correctly, have reasonable values etc. > TBH you could add accessor methods to a dict just as easily ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: In code, list.clear doesn't throw error - it's just ignored
On Tue, 15 Nov 2022 at 05:57, Stefan Ram wrote: > > Michael Speer writes: > >Python doesn't care what an expression returns. > > In my English, functions return values, > expression are being evaluated to a value. > The evaluation of a function yields or > produces a value. Expressions do not return, > because they are not functions. > Well, you can dispute the terminology all you like, but there's no real difference between function calls and other types of expression. They're all just expressions and they can be combined arbitrarily. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: In code, list.clear doesn't throw error - it's just ignored
On Mon, 14 Nov 2022 at 18:00, Greg Ewing wrote: > > On 14/11/22 3:13 pm, MRAB wrote: > > But if it's an expression where it's expecting a statement and it's not > > a call, then it's probably a bug. > > The key word there is "probably". If there's any chance it > could be not a bug, it can't be an error. At most it should > be a warning, and that's what linters are for. I wouldn't > like the core interpreter to be producing a bunch of warnings > for things like this. > Notably, linters can be taught about more complex idioms, like: try: raw_input except NameError: raw_input = input which is an easy way to make polyglot Py2/Py3 code that handles the presence/absence of a particular name. Or, similarly: try: os.sendfile except AttributeError: ... # cope with sendfile not being available When os.sendfile exists, it's a completely useless expression. When it doesn't, it's an error. But the difference between those two is crucial. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: In code, list.clear doesn't throw error - it's just ignored
On Mon, 14 Nov 2022 at 13:18, MRAB wrote: > > On 2022-11-14 00:55, Greg Ewing wrote: > > On 14/11/22 1:31 pm, Jon Ribbens wrote: > >> On 2022-11-13, DFS wrote: > >>> But why is it allowed in the first place? > >> > >> Because it's an expression, and you're allowed to execute expressions. > > > > To put it a bit more clearly, you're allowed to evaluate > > an expression and ignore the result. > > > But if it's an expression where it's expecting a statement and it's not > a call, then it's probably a bug. Maybe, but I'd be dubious of making it that simplistic. For instance, which of these is more likely to be useless? spam or ham() spam() or ham Syntactically, both of them are 'or' expressions, but one of them is almost certainly unnecessary, while the other is just an oddly-written conditional statement and most definitely useful. In any case, this is the job of linters, NOT the language itself. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: In code, list.clear doesn't throw error - it's just ignored
On Mon, 14 Nov 2022 at 11:53, DFS wrote: > > On 11/13/2022 5:20 PM, Jon Ribbens wrote: > > On 2022-11-13, DFS wrote: > >> In code, list.clear is just ignored. > >> At the terminal, list.clear shows > >> > >> > >> > >> in code: > >> x = [1,2,3] > >> x.clear > >> print(len(x)) > >> 3 > >> > >> at terminal: > >> x = [1,2,3] > >> x.clear > >> > >> print(len(x)) > >> 3 > >> > >> > >> Caused me an hour of frustration before I noticed list.clear() was what > >> I needed. > >> > >> x = [1,2,3] > >> x.clear() > >> print(len(x)) > >> 0 > > > > If you want to catch this sort of mistake automatically then you need > > a linter such as pylint: > > > >$ cat test.py > >"""Create an array and print its length""" > > > >array = [1, 2, 3] > >array.clear > >print(len(array)) > >$ pylint -s n test.py > >* Module test > >test.py:4:0: W0104: Statement seems to have no effect > > (pointless-statement) > > > Thanks, I should use linters more often. > > But why is it allowed in the first place? > > I stared at list.clear and surrounding code a dozen times and said > "Looks right! Why isn't it clearing the list?!?!" > > 2 parens later and I'm golden! > No part of it is invalid, so nothing causes a problem. For instance, you can write this: >>> 1 And you can write this: >>> 1 + 2 And you can write this: >>> print(1 + 2) But only one of those is useful in a script. Should the other two be errors? No. But linters WILL usually catch them, so if you have a good linter (especially built into your editor), you can notice these things. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Persisting functions typed into the shell
On Sun, 13 Nov 2022 at 05:48, Stefan Ram wrote: > So much for the topic of "In Python, /everything/ is an > object"! There seem to be first and second-class objects: > Shelveable and non-shelveable objects. > That's a bit unfair. Everything IS an object, but not all objects can be treated the same way. Complex numbers can't be compared for greater-than/less-than, but they are still first-class objects. Is it acceptable to you if the reconstituted function doesn't have source code (which will affect tracebacks and such), as long as it behaves the same way? If so, save the __code__.co_code attribute, and build a new function from that. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Argument name should be lowercase
On Sat, 12 Nov 2022 at 06:42, Stefan Ram wrote:
>
> "Weatherby,Gerard" writes:
> >I'd personally find it weird to see an all-cap parameter
>
> In the source code of the standard library, all-caps
> notation is used for a parameter name sometimes
> if that parameter name is "URL" or sometimes
> when it is being initialized from an all-caps name as in:
>
> |def __del__(self, _warn=warnings.warn, RUN=RUN):
> |if self._state == RUN:
> ...
> from "pool.py".
>
> ("RUN=RUN" makes RUN have the value "RUN" had
> at the time of the definition of the function.)
>
There are a few reasons to do that "snapshot" trick. The most common
is performance, but in this case, my guess is that (since it's a
__del__ method) it's to ensure that the objects are still referenced
when this function is called - to stop them getting disposed of
prematurely. Very rare, but important occasionally.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: What's tkinter doing in \Lib\site-packages\future\moves ?
On Tue, 8 Nov 2022 at 15:12, DFS wrote: > > 3.9.13 > My guess? Because you can "import tkinter" in Py3 but "import Tkinter" in Py2. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How to manage python shebang on mixed systems?
On Tue, 8 Nov 2022 at 08:44, Chris Green wrote: > > Barry Scott wrote: > > > > > > > On 7 Nov 2022, at 09:28, Chris Green wrote: > > > > > > Chris Green wrote: > > >>> 3: with your pseudo "python3" script in place, make all the scripts use > > >>> the "#!/usr/bin/env python3" shebang suggested above. > > >>> > > >> Yes, that sounds a good plan to me, thanks Cameron. > > >> > > > Doesn't '#!/usr/bin/env python3' suffer from the same problem as > > > '#!/usr/bin/python3' in the sense that the env executable might not be > > > in /usr/bin? > > > > env is always available as /usr/bin/env - I think its spec'ed in posix that > > way. > > > > The only reason that things are in /bin are for systems that need a subset > > of > > programs to boot the system to point it can mount /usr. env is not going to > > be > > needed for that use case. > > > Given that the problem system is running a very old Linux I'm not sure > what chance there is that it's fully posix compliant. > > If using "#!/usr/bin/env python3" is a way of avoiding problems if > python3 isn't in /usr/bin then why is it any better depending on env > being in /usr/bin. > I frequently have python3 in /usr/local/bin instead. Sometimes in other places, depending on how many of them I've installed and from where. Using /usr/bin/env python3 ensures that it does the same as the "python3" command even if there's a venv active. (And yes, sometimes that's good, sometimes bad.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How to manage python shebang on mixed systems?
On Tue, 8 Nov 2022 at 06:12, Thomas Passin wrote: > > The problem is that some Linux systems - I think - still use Python2 to > perform various system-related tasks. When they call "python", they > need it to be Python2. If someone has a system like that, they can't > have the "python" command run Python3. > If currently-supported versions of any Linux distributions don't at least make a python3 command available, I would be very surprised. But when you don't have root access, it's not easy to install, even from repositories. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Operator: inappropriate wording?
On Thu, 3 Nov 2022 at 10:17, elas tica wrote: > > Le lundi 31 octobre 2022 à 22:18:57 UTC+1, Chris Angelico a ecrit : > > Wording is hard. Just ask the SQL standard whether NULL is a value. > > > > Indeed, but I think our problem here is simpler ;) > > One could for example omit the incorrect term "operator" while remaining > unambiguous. This would give: Yep. The word "operator" is incorrect when referring to Python's comma (in contrast to, say, C, where the comma actually *is* an operator); and from my understanding, the docs have already been updated to fix this. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: an oop question
On Sat, 5 Nov 2022 at 02:18, Greg Ewing wrote: > > On 4/11/22 12:50 am, Chris Angelico wrote: > > In Python, everything is an object. Doesn't that equally mean that > > Python is purely OOP? > > Depends on what you mean by "purely oop". To me it suggests a > language in which dynamically-dispatched methods are the only > form of code. Python is not one of those, because it has > stand-alone functions. > Define "stand-alone". In Java, all code has to be associated with a class file, but they can be static methods. A Java-like language in which classes are themselves first-class objects (and thus static methods are instance methods on the class) would, in a sense, have nothing but dynamically-dispatched methods as the only form of code. It wouldn't be materially different from regular Java though (at least, not for the sake of this purity; having first-class classes would probably have other benefits). Pike code, like Java code, is always associated with an object or a program (Pike's name for a class). However, built-in functions (implemented in C) can stand entirely alone. Would Pike become "purely OOP" if all standalone builtins were deemed to be methods of some global object? It wouldn't materially change anything. Maybe it's one of those terms that is useless for actual coding (because practicality beats purity), but good for discussions? Good for arguments, at least. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: an oop question
On Sat, 5 Nov 2022 at 02:21, Greg Ewing wrote: > > > r...@zedat.fu-berlin.de (Stefan Ram) writes [that Barbara Liskov said]: > > > >> |If for each object o1 of type S there is an object o2 of > >> |type T such that for all programs P defined in terms of T, > >> |the behavior of P is unchanged when o1 is substituted for o2 > >> |then S is a subtype of T. > > That seems overly restrictive, because it wouldn't allow S to > override a method of T and make it do something different -- > which we do all the time in practice. > > >> Class contracts must hold for subclasses. > > That sounds like a much better way of saying it! > Yeah, I would agree with that latter definition. The trouble is that few programs - and fewer programmers - really define which parts are contracts, so it's much easier to say "a subclass behaves just like the superclass(es) do(es)" - but it would be more accurate to qualify that with "in the ways that the superclass(es) define as standard behaviour". ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: typing: property/setter and lists? [RESOLVED ERRATA]
On Fri, 4 Nov 2022 at 05:48, Paulo da Silva wrote: > > Às 05:32 de 03/11/22, Paulo da Silva escreveu: > > Às 03:24 de 03/11/22, Paulo da Silva escreveu: > >> Hi! > >> > >> And a typing problem again!!! > >> ___ > >> class C: > >> def __init__(self): > >> self.__foos=5*[0] > >> > >> @property > >> def foos(self) -> list[int]: > >> return self.__foos > >> > >> @foos.setter > >> def foos(self,v: int): > >> self.__foos=[v for __i in self.__foos] > >> > >> c=C() > >> c.foos=5 > >> print(c.foos) > >> ___ > >> > >> mypy gives the following error: > >> error: Incompatible types in assignment (expression has type "int", > >> variable has type "List[int]") > >> > >> How do I turn around this? > >> > > Changing def foos(self) -> list[int]: to > > def foos(self) -> Union[list[int]]: > I meant, of course, > def foos(self) -> Union[list[int],int]: > Ohhh! I thought this was triggering a strange quirk of the checker in some way... ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: an oop question
On Fri, 4 Nov 2022 at 05:21, Julieta Shem wrote:
>
> Chris Angelico writes:
>
> > On Thu, 3 Nov 2022 at 21:44, Alan Gauld wrote:
> >> Also Python is not a purely OOP language, in that you can write
> >> functional and procedural code in Python if you wish. In
> >> Smalltalk thats notionally impossible because everything
> >> is an object. And all programming statements are messages
> >> to objects.
> >
> > In Python, everything is an object. Doesn't that equally mean that
> > Python is purely OOP?
>
> I think Alan Gauld pointed out that even syntax is an object in
> Smalltalk --- or was. An if-statement in Python is not an object.
Okay, fair; although I would be highly surprised if syntax is actually
an object ("flow control is an object", I would believe, though). Is
the concept "pass this message to this object" an object? Is the
message itself an object? Is it objects all the way down?
At some point, any language with objects in it is "object oriented" to
some extent, and after that, it's all a spectrum. Some people claim
that Java is "more object-oriented" than Python because all Java code
has to be in a class, and others counteract by saying that Python is
"more object-oriented" because every value in Python is a subclass of
object and has a type which is a subclass of type. I'm sure that
someone somewhere has claimed that Python "isn't object-oriented" on
the basis that len(x) is the WRONG way to put it, and it should be
x.length() instead.
On second thoughts, it's not a single spectrum, it's a
multidimensional thing that's so tangled up that it guarantees that
people can happily debate for years to come.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: typing: property/setter and lists? [RESOLVED]
On Fri, 4 Nov 2022 at 05:03, Paulo da Silva wrote: > Changing def foos(self) -> list[int]: to > def foos(self) -> Union[list[int]]: > fixes the problem. > Not so elegant, however! Wait, what?! Union[X, Y] means "X or Y" Union[X] means "X, but don't complain if it's a @property". Is that how it goes? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: an oop question
On Thu, 3 Nov 2022 at 21:44, Alan Gauld wrote: > Also Python is not a purely OOP language, in that you can write > functional and procedural code in Python if you wish. In > Smalltalk thats notionally impossible because everything > is an object. And all programming statements are messages > to objects. In Python, everything is an object. Doesn't that equally mean that Python is purely OOP? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Operator: inappropriate wording?
On Tue, 1 Nov 2022 at 08:15, elas tica wrote: > > Le mercredi 26 octobre 2022 à 22:12:59 UTC+2, Weatherby,Gerard a ecrit : > > No. If the docs say in one place a comma is not an operator, they shouldn’t > > call it an operator in another place. > > > > I’ve submitted a pull request https://github.com/python/cpython/pull/98736 > > -- we’ll have to see what The Powers That Be think. > > > Thanks for the (merged) pull request about the "comma operator"! > > I return to the last two quotes in the Reference Document regarding these > so-called "assignment operators". > > The entry in the glossary explains that the comma symbol is not an operator. > Well, I just realized that this same entry also explains that the = symbol is > not an operator, as you can see by reading the end of their response: > > The same is true of the various assignment operators (=, += etc). They are > not truly operators but syntactic delimiters in assignment statements. > > (glossary entry link: > https://docs.python.org/3/faq/programming.html#what-s-up-with-the-comma-operator-s-precedence) > > Talking about an assignment operator in Python is even more confusing > because, since Python 3.8, there is a real assignment operator, namely the > walrus operator. As explained above, the correct expression would be > "assignement delimiter" or "assignement statement" or "assignement symbol". > Wording is hard. Just ask the SQL standard whether NULL is a value. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: an oop question
On Mon, 31 Oct 2022 at 14:38, Julieta Shem wrote:
>
> Chris Angelico writes:
>
> > The most straight-forward way to represent this concept in an
> > object-oriented way is subclassing.
> >
> > class Stack:
> > ... # put whatever code is common here
> >
> > class Empty(Stack):
> > ... # put Empty-specific code here, possibly overriding Stack methods
> >
> > class Pair(Stack):
> >... # ditto, overriding or augmenting as needed
> >
> > This way, everything is an instance of Stack, but they are still
> > distinct types for when you need to distinguish.
>
> Can you provide a small example? I can't see what you mean, but it
> seems interesting.
Sure. The easiest way would be to take your existing Empty and Pair
classes, have them subclass Stack, and don't bother putting any code
at all into Stack. Then construct an Empty and a few Pairs, and what
you'll see is that all of them are also instances of Stack.
After that, it's really a question of what you expect to be able to do
with either an Empty or a Pair. Anything that should be possible with
both types (that is, anything that should be possible with either
variant of Stack) should get moved into the Stack type, while anything
that is specific to one or the other stays in its own class. So here's
a very very simple example:
class Stack:
def prepend(self, other):
return Pair(other, self)
class Empty(Stack):
def is_last(self):
return True
def get_current(self):
raise ValueError("Stack empty")
def get_next(self):
raise ValueError("Stack empty")
class Pair(Stack):
def __init__(self, item1, item2):
self.item1 = item1
self.item2 = item2
def get_current(self):
return self.item1
def get_next(self):
return self.item2
def is_last(self):
return isinstance(self.item2, Empty)
With this setup, you can build a stack by prepending items onto an
Empty endpoint, and can iterate over it with the get_current and
get_next methods. (Making this actually iterable, so that it works
with a Python 'for' loop, would be a good exercise.)
In this example, there isn't much code in the Stack class. But you
could easily add more, and it would apply to both Empty and Pair.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: A typing question
On Mon, 31 Oct 2022 at 09:39, dn wrote: > > On 31/10/2022 06.06, Stefan Ram wrote: > > Paulo da Silva writes: > >> Is there anything to do without loosing my script structure and usual > >> practice? > > > >to lose (losing): to stop having something > >to loose (loosing): to let or make loose (see next line) > >loose (adj.): not firmly attached/tied/fastened/controlled > >to loosen: similar to "to loose" > > > Hay, your write*! > > Well done. There's many a native-speaker who doesn't know the > distinction, or doesn't care about accuracy. > I'm curious to what extent sloppy English correlates with sloppy code. Do people care about learning proper Python but not about proper English, or do they think there's no such thing as proper English just because there's no English Steering Council? A lot of people seem to treat English the way web browsers treat HTML - as long as you can make some sense out of it, it's good enough. Some nerds treat English the way the W3C treats HTML - there actually is a standard and everything has defined rules. I know which camp I prefer to communicate with. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: an oop question
On Mon, 31 Oct 2022 at 09:05, Julieta Shem wrote: > > Julieta Shem writes: > > [...] > > >> . If you should, however, be talking about the new "type hints": > >> These are static and have "Union", for example, "Union[int, str]" > >> or "int | str". > > > > I ended up locating such features of the language in the documentation, > > but I actually am not interested in declaring the type to the compiler > > (or to the reader). > > > > I was looking for a solution like yours --- thank you! ---, although I > > was hoping for handling that situation in the construction of the Stack > > object, which was probably why I did not find a way out. Right now I'm > > looking into __new__() to see if it can somehow produce one type or > > another type of object depending on how the user has invoked the class > > object. > > > > Terminology. By ``invoking the class object'' I mean expressions such > > as Class1() or Class2(). ``Class1'' represents the object that > > represents the class 1. Since the syntax is that of procedure > > invokation, I say ``invoking the class object''. > > An experiment. What's my definition of Stack? It's either Empty or > Pair, so it's a union. So let us create two inner classes (that is, > inner to Stack) and make them behave just like Empty and Pair. Using > __new__(), we can produce a Stack object that is sometimes Empty() and > sometimes Pair(...). > The most straight-forward way to represent this concept in an object-oriented way is subclassing. class Stack: ... # put whatever code is common here class Empty(Stack): ... # put Empty-specific code here, possibly overriding Stack methods class Pair(Stack): ... # ditto, overriding or augmenting as needed This way, everything is an instance of Stack, but they are still distinct types for when you need to distinguish. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: UTF-8 and latin1
On Wed, 26 Oct 2022 at 05:09, Barry Scott wrote: > > > > > On 25 Oct 2022, at 11:16, Stefan Ram wrote: > > > > r...@zedat.fu-berlin.de (Stefan Ram) writes: > >> You can let Python guess the encoding of a file. > >> def encoding_of( name ): > >> path = pathlib.Path( name ) > >> for encoding in( "utf_8", "cp1252", "latin_1" ): > >> try: > >> with path.open( encoding=encoding, errors="strict" )as file: > > > > I also read a book which claimed that the tkinter.Text > > widget would accept bytes and guess whether these are > > encoded in UTF-8 or "ISO 8859-1" and decode them > > accordingly. However, today I found that here it does > > accept bytes but it always guesses "ISO 8859-1". > > The best you can do is assume that if the text cannot decode as utf-8 it may > be 8859-1. > Except when it's Windows-1252. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beautiful Soup - close tags more promptly?
On Wed, 26 Oct 2022 at 04:59, Tim Delaney wrote: > > On Mon, 24 Oct 2022 at 19:03, Chris Angelico wrote: >> >> >> Ah, cool. Thanks. I'm not entirely sure of the various advantages and >> disadvantages of the different parsers; is there a tabulation >> anywhere, or at least a list of recommendations on choosing a suitable >> parser? > > > Coming to this a bit late, but from my experience with BeautifulSoup and HTML > produced by other people ... > > lxml is easily the fastest, but also the least forgiving. > html.parer is middling on performance, but as you've seen sometimes makes > mistakes. > html5lib is the slowest, but is most forgiving of malformed input and edge > cases. > > I use html5lib - it's fast enough for what I do, and the most likely to > return results matching what the author saw when they maybe tried it in a > single web browser. Cool cool. It sounds like html5lib should really be the recommended parser for HTML, unless performance or dependency reduction is important enough to change your plans. (But only for HTML. For XML, lxml would still be the right choice.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beautiful Soup - close tags more promptly?
On Tue, 25 Oct 2022 at 09:34, Peter J. Holzer wrote: > > One thing I find quite interesting, though, is the way that browsers > > *differ* in the face of bad nesting of tags. Recently I was struggling > > to figure out a problem with an HTML form, and eventually found that > > there was a spurious tag way up higher in the page. Forms don't > > nest, so that's invalid, but different browsers had slightly different > > ways of showing it. > > Yeah, mismatched form tags can have weird effects. I don't remember the > details but I scratched my head over that one more than once. > Yeah. I think my weirdest issue was one time when I inadvertently had a element (with a form inside it) inside something else with a form (because the was missing). Neither "dialog inside main" nor "form in dialog separate from form in main" is a problem, and even "oops, missed a closing form tag" isn't that big a deal, but put them all together, and you end up with a bizarre situation where Firefox 91 behaves one way and Chrome (some-version) behaves another way. That was a fun day. Remember, folks, even if you think you ran the W3C validator on your code recently, it can still be worth checking. Just in case. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beautiful Soup - close tags more promptly?
On Tue, 25 Oct 2022 at 04:22, Peter J. Holzer wrote: > There may be several reasons: > > * Historically, some browsers differed in which end tags were actually > optional. Since (AFAIK) no mainstream browser ever implemented a real > SGML parser (they were always "tag soup" parsers with lots of ad-hoc > rules) this sometimes even changed within the same browser depending > on context (e.g. a simple table might work but nested tables woudn't). > So people started to use end-tags defensively. > * XHTML was for some time popular and it doesn't have any optional tags. > So people got into the habit of always using end tags and writing > empty tags as . > * Aesthetics: Always writing the end tags is more consistent and may > look more balanced. > * Cargo-cult: People saw other people do that and copied the habit > without thinking about it. > > > > Are you saying that it's better to omit them all? > > If you want to conserve keystrokes :-) > > I think it doesn't matter. Both are valid. > > > More importantly: Would you omit all the closing tags you can, or > > would you include them? > > I usually write them. Interesting. So which of the above reasons is yours? Personally, I do it for a slightly different reason: Many end tags are *situationally* optional, and it's much easier to debug code when you change/insert/remove something and nothing changes, than when doing so affects the implicit closing tags. > I also indent the contents of an element, so I > would write your example as: > > > > > Hello, world! > > Paragraph 2 > > > Hey look, a third paragraph! > > > > > (As you can see I would also include the body tags to make that element > explicit. I would normally also add a bit of boilerplate (especially a > head with a charset and viewport definition), but I omit them here since > they would change the parse tree) > Yeah - any REAL page would want quite a bit (very few pages these days manage without a style sheet, and it seems that hardly any survive without importing a few gigabytes of JavaScript, but that's not mandatory), but in ancient pages, there's still a well-defined parse structure for every tag sequences. One thing I find quite interesting, though, is the way that browsers *differ* in the face of bad nesting of tags. Recently I was struggling to figure out a problem with an HTML form, and eventually found that there was a spurious tag way up higher in the page. Forms don't nest, so that's invalid, but different browsers had slightly different ways of showing it. (Obviously the W3C Validator was the most helpful tool here, since it reports it as an error rather than constructing any sort of DOM tree.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beautiful Soup - close tags more promptly?
On Tue, 25 Oct 2022 at 02:45, Jon Ribbens via Python-list wrote: > > On 2022-10-24, Chris Angelico wrote: > > On Mon, 24 Oct 2022 at 23:22, Peter J. Holzer wrote: > >> Yes, I got that. What I wanted to say was that this is indeed a bug in > >> html.parser and not an error (or sloppyness, as you called it) in the > >> input or ambiguity in the HTML standard. > > > > I described the HTML as "sloppy" for a number of reasons, but I was of > > the understanding that it's generally recommended to have the closing > > tags. Not that it matters much. > > Some elements don't need close tags, or even open tags. Unless you're > using XHTML you don't need them and indeed for the case of void tags > (e.g. , ) you must not include the close tags. Yep, I'm aware of void tags, but I'm talking about the container tags - in this case, and - which, in a lot of older HTML pages, are treated as "separator" tags. Consider this content: Hello, world! Paragraph 2 Hey look, a third paragraph! Stick a doctype onto that and it should be valid HTML5, but as it is, it's the exact sort of thing that was quite common in the 90s. (I'm not sure when lowercase tags became more popular, but in any case (pun intended), that won't affect validity.) The tag is not a void tag, but according to the spec, it's legal to omit the if the element is followed directly by another element (or any of a specific set of others), or if there is no further content. > Adding in the omitted , , , , and > would make no difference and there's no particular reason to recommend > doing so as far as I'm aware. And yet most people do it. Why? Are you saying that it's better to omit them all? More importantly: Would you omit all the closing tags you can, or would you include them? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beautiful Soup - close tags more promptly?
On Mon, 24 Oct 2022 at 23:22, Peter J. Holzer wrote: > > On 2022-10-24 21:56:13 +1100, Chris Angelico wrote: > > On Mon, 24 Oct 2022 at 21:33, Peter J. Holzer wrote: > > > Ron has already noted that the lxml and html5 parser do the right thing, > > > so just for the record: > > > > > > The HTML fragment above is well-formed and contains a number of li > > > elements at the same level directly below the ol element, not lots of > > > nested li elements. The end tag of the li element is optional (except in > > > XHTML) and li elements don't nest. > > > > That's correct. However, parsing it with html.parser and then > > reconstituting it as shown in the example code results in all the > > tags coming up right before the , indicating that the > > tags were parsed as deeply nested rather than as siblings. > > Yes, I got that. What I wanted to say was that this is indeed a bug in > html.parser and not an error (or sloppyness, as you called it) in the > input or ambiguity in the HTML standard. I described the HTML as "sloppy" for a number of reasons, but I was of the understanding that it's generally recommended to have the closing tags. Not that it matters much. > > which html5lib seems to be doing fine. Whether > > it has other issues, I don't know, but I guess I'll find out > > The link somebody posted mentions that it's "very slow". Which may or > may not be a problem when you have to parse 9000 files. But if it does > implement HTML5 correctly, it should parse any file the same as a modern > browser does (maybe excluding quirks mode). > Yeah. TBH I think the two-hour run time is primarily dominated by network delays, not parsing time, but if I had a service where people could upload HTML to be parsed, that might affect throughput. For the record, if anyone else is considering html5lib: It is likely "fast enough", even if not fast. Give it a try. (And I know what slow parsing feels like. Parsing a ~100MB file with a decently-fast grammar-based lexer takes a good while. Parsing the same content after it's been converted to JSON? Fast.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beautiful Soup - close tags more promptly?
On Mon, 24 Oct 2022 at 21:33, Peter J. Holzer wrote: > Ron has already noted that the lxml and html5 parser do the right thing, > so just for the record: > > The HTML fragment above is well-formed and contains a number of li > elements at the same level directly below the ol element, not lots of > nested li elements. The end tag of the li element is optional (except in > XHTML) and li elements don't nest. That's correct. However, parsing it with html.parser and then reconstituting it as shown in the example code results in all the tags coming up right before the , indicating that the tags were parsed as deeply nested rather than as siblings. In order to get a successful parse out of this, I need something which sees them as siblings, which html5lib seems to be doing fine. Whether it has other issues, I don't know, but I guess I'll find out it's currently running on the live site and taking several hours (due to network delays and the server being slow, so I don't really want to parallelize and overload the thing). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Beautiful Soup - close tags more promptly?
On Mon, 24 Oct 2022 at 18:43, Roel Schroeven wrote: > > Op 24/10/2022 om 4:29 schreef Chris Angelico: > > Parsing ancient HTML files is something Beautiful Soup is normally > > great at. But I've run into a small problem, caused by this sort of > > sloppy HTML: > > > > from bs4 import BeautifulSoup > > # See: https://gsarchive.net/gilbert/plays/princess/tennyson/tenniv.htm > > blob = b""" > > > > 'THERE sinks the nebulous star we call the Sun, > > If that hypothesis of theirs be sound,' > > Said Ida;' let us down and rest:' and we > > Down from the lean and wrinkled precipices, > > By every coppice-feather'd chasm and cleft, > > Dropt thro' the ambrosial gloom to where below > > No bigger than a glow-worm shone the tent > > Lamp-lit from the inner. Once she lean'd on me, > > Descending; once or twice she lent her hand, > > And blissful palpitations in the blood, > > Stirring a sudden transport rose and fell. > > > > """ > > soup = BeautifulSoup(blob, "html.parser") > > print(soup) > > > > > > On this small snippet, it works acceptably, but puts a large number of > > tags immediately before the . On the original file (see > > link if you want to try it), this blows right through the default > > recursion limit, due to the crazy number of "nested" list items. > > > > Is there a way to tell BS4 on parse that these elements end at > > the next , rather than waiting for the final ? This would > > make tidier output, and also eliminate most of the recursion levels. > > > Using html5lib (install package html5lib) instead of html.parser seems > to do the trick: it inserts right before the next , and one > before the closing . On my system the same happens when I don't > specify a parser, but IIRC that's a bit fragile because other systems > can choose different parsers of you don't explicity specify one. > Ah, cool. Thanks. I'm not entirely sure of the various advantages and disadvantages of the different parsers; is there a tabulation anywhere, or at least a list of recommendations on choosing a suitable parser? I'm dealing with a HUGE mess of different coding standards, all the way from 1990s-level stuff (images for indentation, tables for formatting, and ) up through HTML4 (a good few of the pages have at least some tags and declare their encodings, mostly ISO-8859-1 or similar), to fairly modern HTML5. There's even a couple of pages that use frames - yes, the old style with a block in case the browser can't handle it. I went with html.parser on the expectation that it'd give the best "across all standards" results, but I'll give html5lib a try and see if it does better. Would rather not try to use different parsers for different files, but if necessary, I'll figure something out. (For reference, this is roughly 9000 HTML files that have to be parsed. Doing things by hand is basically not an option.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Typing: Is there a "cast operator"?
On Mon, 24 Oct 2022 at 14:15, Dan Stromberg wrote: > I've found that mypy understands simple assert statements. > > So if you: > if f is not None: > assert f is not None > os.write(f, ...) > > You might be in good shape. Why can't it simply understand the if statement? I'm not a fan of coddling a type system like this. The entire point of type checking is to help you find bugs more efficiently, so if you have to repeat yourself every time you do these kinds of checks just so that mypy is satisfied, that's counter-productive (case in point: what happens if you say "if fn is not None: assert f is not None"? Now you've introduced a bug just to deal with the type system). ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Beautiful Soup - close tags more promptly?
Parsing ancient HTML files is something Beautiful Soup is normally great at. But I've run into a small problem, caused by this sort of sloppy HTML: from bs4 import BeautifulSoup # See: https://gsarchive.net/gilbert/plays/princess/tennyson/tenniv.htm blob = b""" 'THERE sinks the nebulous star we call the Sun, If that hypothesis of theirs be sound,' Said Ida;' let us down and rest:' and we Down from the lean and wrinkled precipices, By every coppice-feather'd chasm and cleft, Dropt thro' the ambrosial gloom to where below No bigger than a glow-worm shone the tent Lamp-lit from the inner. Once she lean'd on me, Descending; once or twice she lent her hand, And blissful palpitations in the blood, Stirring a sudden transport rose and fell. """ soup = BeautifulSoup(blob, "html.parser") print(soup) On this small snippet, it works acceptably, but puts a large number of tags immediately before the . On the original file (see link if you want to try it), this blows right through the default recursion limit, due to the crazy number of "nested" list items. Is there a way to tell BS4 on parse that these elements end at the next , rather than waiting for the final ? This would make tidier output, and also eliminate most of the recursion levels. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Portable executable on OSX
On Fri, 21 Oct 2022 at 08:50, Thomas Passin wrote: > > "Portable executable" usually means that the program resides on > removable media, like a USB stick. You can go to a computer, plug the > stick in, and run the program. If it's Python, then the installation on > the removable medium needs to set up all the paths and directories > correctly before actually running Python. That would typically be done > with batch files setting paths and environmental variables. > > I got this working for Python on Windows some years ago. Here is the > setup batch file I used - it gets executed when the user types "pyth37": > > @echo off > setlocal > : Find effective drive for this file. > set ed=%~d0 > path %ed%\python37\Scripts;%ed%\python37;%PATH% > set PYTHONUSERBASE=%ed%\user\python > set HOME=%ed%\user\python > call python %* > endlocal > > It might need to be be more complex on MacOS, but it gives you the idea. > The odd-looking line "set ed=%~d0" is a Windows-specific way to get > the drive of the command file being run. > Basic idea looks sound. Might actually be _easier_ on OSX, since it's Unix-like and you should be able to depend on /bin/bash. The notation `dirname $0` should give you the path to the current script, from which everything else can be calculated. (Caveat: Never actually done this on a Mac, and only did cursory web searching to check that it wasn't a ridiculous idea.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Wed, 19 Oct 2022 at 12:01, Peter J. Holzer wrote: > > On 2022-10-17 09:25:00 +0200, Karsten Hilbert wrote: > > > which had special combinations for all the BASIC keywords). And if you > > > go this way, why not go a step further and dissociate the program from > > > its linear text representation? Add footnotes, different views, > > > hyperlinks, format mathematical expressions like formulas, etc. > > > > http://literateprogramming.com/ > > Right. That's one of the inspirations for my comment. > > But literate programming is of course still very much rooted in the > "linear text representation" paradigm. You have one definite source > which is a linear text. > > In a world of IDEs, databases and hypertext that's probably not the best > we can do. As Raymond Hettinger would say, "there must be a better way". > > It would be very different from mainstream programming languages, > however. And ideally you would want it to integrate with a lot of other > infrastructure. So that alone might make it a non-starter, even if it > was really good (which realistically early iterations wouldn't be). > There are special-purpose languages like Scratch which are not simple text in that form. My Twitch channel bot has a command executor whose language, if you call it that, is basically JSON - and the way to edit those commands is very Scratch-inspired. These are not general-purpose programming languages, but for what they do, they can be very useful. It's a great way to put together incredibly high level primitives that tie in well with the system they're in. (Imagine programming a model train set, with primitives like "change the points here to go straight" and "advance train #32 to the next red signal".) I'm not sure how you'd make that useful for general-purpose programming, but there's definitely a LOT of value in non-textual languages for certain situations. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Quick question about CPython interpreter
On Tue, 18 Oct 2022 at 03:51, Stefan Ram wrote: > > MRAB writes: > >It can't optimise that because, say, 'print' could've been bound to a > >function that rebinds 'str'. > > It would be possible to find out whether a call of a function > named "print" is to the standard function, but the overhead > to do this in the end might slow down the execution. > > In general, it can be possible that there could be optimizer > stages after compilation. So, one might write a small micro- > benchmark to be sure. > You'd also have to ensure that the stringification of the ID doesn't change (which it can it it isn't a core data type), and the easiest way to do THAT is to call str() on the ID every time and see if it's the same... ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Optional arguments in a class behave like class attributes.
On Tue, 18 Oct 2022 at 01:39, Abderrahim Adrabi wrote: > So, these default values behave like class attributes, here is a demo: > > # Using a list - > class GameOne: > def __init__(self, games = []) -> None: > self.games = games > This makes the default be a single list, the same list for all of them. If you want the default to be "construct a new list", you'll need something else: def __init__(self, games=None): if games is None: games = [] There is an open proposal to allow this syntax, which would do what you want: def __init__(self, games=>[]): However, this is not part of any current version of Python. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 17 Oct 2022 at 16:36, Antoon Pardon wrote: > > > > Op 17/10/2022 om 04:01 schreef Chris Angelico: > > On Mon, 17 Oct 2022 at 10:46, wrote: > >> My point Chris was that you can have a conversation where you are exploring > >> and not proposing. Brainstorming, perhaps. > > And my point is that either a proposal is a serious one that can > > expect serious discussion, or it isn't. Yes, I'm aware that it wasn't > > you who backpedalled as soon as any criticism was posted, but your > > caveat comes to the same thing - if you're trying to avoid serious > > criticism, you have to not post an idea. > > Your reaction was not serious criticisme but a belligerent reaction. > You made it clear you wanted a fight. I choose not to enter that > fight. > Well, if reacting strongly to a proposal to break LITERALLY EVERY PYTHON PROGRAM EVER counts as not a serious criticism but a belligerent reaction, then there's not really a lot of point discussing further. Bye. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 17 Oct 2022 at 10:46, wrote: > > My point Chris was that you can have a conversation where you are exploring > and not proposing. Brainstorming, perhaps. And my point is that either a proposal is a serious one that can expect serious discussion, or it isn't. Yes, I'm aware that it wasn't you who backpedalled as soon as any criticism was posted, but your caveat comes to the same thing - if you're trying to avoid serious criticism, you have to not post an idea. > So my POINT (and I repeat NOT a suggestion) is that I can IMAGINE ways to > add a feature to a loop, such as an extra optional argument that is called > if the loop exits from the bottom. The code you now put in the ELSE clause > might have to be in the lambda or whatever. That may not be a good fit for > Python. If you're inventing a completely new language, you can do whatever you like, but it's not very practical to discuss small features when there's no language to discuss them in. So are you discussing this as a Python feature, or just saying "hey, in a vacuum, we could do this", which is vacuously true? > What may aggravate you is that lots of people keep telling you that the ELSE > on a loop feature is not intuitive to many, sometimes even after it is > explained. I know it's unintuitive. That doesn't stop it being useful. > My suggestion is you should deal with that and not take it out on others. > Live and let live. That's unrelated. Sorry to disappoint you. > This forum may be about Python but not exclusively. I personally often enjoy > hearing how some other system does something similar, such as discussions on > how and whether Python should allow an underscore in static numbers given > other languages do so, not always identically. We can learn from such > comparisons, for good and also for bad. That's correct, but usually there's at least SOME context. And if none is given, is it or is it not reasonable to assume that people are talking about Python? > But if it makes you happy, take me off this list! I have so many things I > need to do and free up time for. I'm not in a position to do that, and my point was to show how ridiculous empty proposals are :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 17 Oct 2022 at 08:22, wrote: > I had another crazy thought that I AM NOT ASKING anyone to do. OK? > Here's another proposal: Let's ban you from this mailing list. Don't worry, I AM NOT ASKING anyone to do it. OK? Do you see how ridiculous and pointless it is to have proposals with that kind of caveat? Make serious proposals that we can discuss reasonably, don't make fake proposals and hide behind a caveat. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 17 Oct 2022 at 03:57, Antoon Pardon wrote: > > > Op 16/10/2022 om 17:05 schreef Chris Angelico: > > On Sun, 16 Oct 2022 at 22:47, Antoon Pardon wrote: > >> Why would I need good luck? I expressed an idea and you didn't like it. > >> That won't affect my life in a meaningful way. > > Well, with that attitude, it's not going to affect anyone else's life > > either, so go ahead, carry on. > > What attitude? I just floated a little idea. It was not meant/expected to > affect anyone else's life. So why do you react as if it was? > You expressed an idea that you would like to see implemented in Python, and part of that idea was that people would be *obliged* to write their code using non-ASCII keywords. If that were to be implemented, it absolutely WOULD affect many people's lives. So if you're saying that your idea was not meant to affect anyone else's life, you are saying that you floated the idea fully intending for it to be ignored, which makes me wonder why you posted it in the first place. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Sun, 16 Oct 2022 at 22:47, Antoon Pardon wrote: > > > > Op 16/10/2022 om 13:03 schreef Chris Angelico: > > On Sun, 16 Oct 2022 at 21:19, Antoon Pardon wrote: > > > >> My idea would be to reserve different unicode blocks for the keywords > >> and the identifiers. e.g. We could reserve the mathematical alphanumeric > >> block for keywords and all other letters and numbers for identifiers. > >> Doing so would allow extenting the keywords without breaking programs > >> that already use that combination as an identifier. > > Python currently defines identifiers as follows: > > https://docs.python.org/3/reference/lexical_analysis.html#identifiers > > > > Briefly, what it means is that (aside from some backward compatibility > > special cases) an identifier contains letters, numbers, and connector > > punctuation, and must not start with a number. It's not by blocks, > > it's by types. > > > > It's way WAY too late to change what's allowed for identifiers, as you > > will potentially be breaking programs that use the current rules. > > So? Python has broken backward compatibility before. The cost could > be acceptable. How many programs do you estimated use the mathematical > alphanumeric block for an identifier at this moment? > Why would I need good luck? I expressed an idea and you didn't like it. > That won't affect my life in a meaningful way. Well, with that attitude, it's not going to affect anyone else's life either, so go ahead, carry on. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Sun, 16 Oct 2022 at 21:19, Antoon Pardon wrote: > > Op 16/10/2022 om 00:50 schreef avi.e.gr...@gmail.com: > > This has been discussed so often precisely because I swear NO CHOICE of > > keyword would satisfy everybody! Most languages start with designated > > keywords and some reserve a few for later use. But then things can get > > frozen in place to avoid breaking existing programs or break older > > compilers/interpreters. > > > > Some languages use techniques to extend themselves more harmlessly such as > > creating a singleton object that has content that can be regular data as in > > math.pi, or functions/methods or new ides like "Symbols" that allow all > > kinds of extensions to the language in a fairly harmless way as no older > > program would likely have used features that did not exist. > > > > That might not easily solve this problem. But I wonder if reserving some > > kind of prefix might help, so anything like extension.0nNoBreak could be > > added to a loop as a final clause and be treated as a non-key keyword of > > sorts. > > My idea would be to reserve different unicode blocks for the keywords > and the identifiers. e.g. We could reserve the mathematical alphanumeric > block for keywords and all other letters and numbers for identifiers. > Doing so would allow extenting the keywords without breaking programs > that already use that combination as an identifier. Python currently defines identifiers as follows: https://docs.python.org/3/reference/lexical_analysis.html#identifiers Briefly, what it means is that (aside from some backward compatibility special cases) an identifier contains letters, numbers, and connector punctuation, and must not start with a number. It's not by blocks, it's by types. It's way WAY too late to change what's allowed for identifiers, as you will potentially be breaking programs that use the current rules. > Python could slowly transition in this direction by first allowing the > current keywords to be in this block. Every new keyword would only be in > that unicode block. If would then be possible to write python code with > this convention but it wouldn't be obligatory. After some time the > python developers could decide to make it obligatory. Obligatory??? Please explain how you intend to convince the entire world that non-ASCII code is an acceptable requirement. Explain to me how you're going to go to every text editor and ensure that it supports easy entry of Python keywords that aren't ASCII. And please explain how this is even better. > I doubt this will idea will get from the ground, but I think it would > allow for a smoother transition into new concepts, as it is no longer a > strugle searching for a keyword that will break as little programs as > possible. Yeah it won't. Good luck though. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Thu, 13 Oct 2022 at 11:23, dn wrote: > # add an extra character within identifier, as if 'new' identifier > 28 assert expected_value == fyibonacci_number > UUU > > # these all trivial SYNTAX errors - could have tried leaving-out a > keyword, but ... Just to be clear, this last one is not actually a *syntax* error - it's a misspelled name, but contextually, that is clearly a name and nothing else. These are much easier to report multiples of, and typical syntax highlighters will do so. Your other two examples were both syntactic discrepancies though. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Thu, 13 Oct 2022 at 11:19, Peter J. Holzer wrote: > > On 2022-10-11 09:47:52 +1100, Chris Angelico wrote: > > On Tue, 11 Oct 2022 at 09:18, Cameron Simpson wrote: > > > > > Consider: > > > > if condition # no colon > > code > > else: > > code > > > > To actually "restart" parsing, you have to make a guess of some sort. > > Right. At least one of the papers on parsing I read over the last few > years (yeah, I really should try to find them again) argued that the > vast majority of syntax errors is either a missing token, a superfluous > token or a combination of the the two. So one strategy with good results > is to heuristically try to insert or delete single tokens and check > which results in the longest distance to the next error. > > Checking multiple possible fixes has its cost, especially since you have > to do that at every error. So you can argue that it is better for > productivity if you discover one error in 0.1 seconds than 10 errors in > 5 seconds. Maybe; but what if you report 10 errors in 5 seconds, but 8 of them are spurious? You've reported two useful errors in a sea of noise. Even if it's the other way around (8 where you nailed it and correctly reported the error, 2 that are nonsense), is it actually helpful? Bear in mind that, if you can discover one syntax error in 0.1 seconds, you can do that check *the moment the user types a key* in the editor (which is more-or-less what happens with most syntax highlighting editors - some have a small delay to avoid being too noisy with error reporting, but same difference). Why report false errors when you can report errors one by one and know that they're true? > > > I grew up with C and Pascal compilers which would _happily_ produce many > > > complaints, usually accurate, and all manner of syntactic errors. They > > > didn't stop at the first syntax error. > > > > Yes, because they work with a much simpler grammar. > > I very much doubt that. Python doesn't have a particularly complicated > grammar, and C certainly doesn't have a particularly simple one. > > The argument that it's impossible in Python (unlike any other language), > because Python is oh so special doesn't hold water. > Never said it's because Python is special; there are a LOT of languages that are at least as complicated. Try giving multiple useful errors when there's a syntactic problem in SQL, for instance. But I do think that Pascal, especially, has a significantly simpler grammar than Python does. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Wed, 12 Oct 2022 at 05:23, Thomas Passin wrote: > > On 10/11/2022 3:10 AM, avi.e.gr...@gmail.com wrote: > > I see resemblances to something like how a web page is loaded and operated. > > I mean very different but at some level not so much. > > > > I mean a typical web page is read in as HTML with various keyword regions > > expected such as ... or ... with things > > often cleanly nested in others. The browser makes nodes galore in some kind > > of tree format with an assortment of objects whose attributes or methods > > represent aspects of what it sees. The resulting treelike structure has > > names like DOM. > > To bring things back to the context of the original post, actual web > browsers are extremely tolerant of HTML syntax errors (including > incorrect nesting of tags) in the documents they receive. They usually > recover silently from errors and are able to display the rest of the > page. Usually they manage this correctly. Having had to debug tiny errors in HTML pages that resulted in extremely weird behaviour, I'm not sure that I agree that they usually manage correctly. Fundamentally, they guess, and guesswork is never reliable. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Tue, 11 Oct 2022 at 18:12, wrote: > > Thanks for a rather detailed explanation of some of what we have been > discussing, Chris. The overall outline is about what I assumed was there but > some of the details were, to put it politely, fuzzy. > > I see resemblances to something like how a web page is loaded and operated. > I mean very different but at some level not so much. > > I mean a typical web page is read in as HTML with various keyword regions > expected such as ... or ... with things > often cleanly nested in others. The browser makes nodes galore in some kind > of tree format with an assortment of objects whose attributes or methods > represent aspects of what it sees. The resulting treelike structure has > names like DOM. Yes. The basic idea of "tokenize, parse, compile" can be used for pretty much any language - even English, although its grammar is a bit more convoluted than most programming languages, with many weird backward compatibility features! I'll parse your last sentence above: LETTERS The SPACE LETTERS resulting SPACE ... you get the idea LETTERS like SPACE LETTERS DOM FULLSTOP # or call this token PERIOD if you're American Now, we can group those tokens into meaningful sets. Sentence(type=Statement, subject=Noun(name="structure", addenda=[ Article(type=The), Adjective(name="treelike"), ]), verb=Verb(type=Being, name="has", addenda=[]), object=Noun(name="name", plural=True, addenda=[ Adjective(phrase=Phrase(verb=Verb(name="like"), object=Noun(name="DOM"), ]), ) Grammar nerds will probably dispute some of the awful shorthanding I did here, but I didn't want to devise thousands of AST nodes just for this :) > To a certain approximation, this tree starts a certain way but is regularly > being manipulated (or perhaps a copy is) as it regularly is looked at to see > how to display it on the screen at the moment based on the current tree > contents and another set of rules in Cascading Style Sheets. Yep; the DOM tree is initialized from the HTML (usually - it's possible to start a fresh tree with no HTML) and then can be manipulated afterwards. > These are not at all the same thing but share a certain set of ideas and > methods and can be very powerful as things interact. Oh absolutely. That's why there are languages designed to help you define other languages. > In effect the errors in the web situation have such analogies too as in what > happens if a region of HTML is not well-formed or uses a keyword not > recognized. And they're horribly horribly messy, due to a few decades of sloppy HTML programmers and the desire to still display the page even if things are messed up :) But, again, there's a huge difference between syntactic errors (like omitting a matching angle bracket) and semantic errors (a keyword not known, like using when you should have used ). In the latter case, you can still build a DOM tree, but you have an unknown element; in the former case, you have to guess at what the author meant, just to get anything going at all. > There was a guy around a few years ago who suggested he would create a > system where you could create a series of some kind of configuration files > for ANY language and his system would them compile or run programs for each > and every such language? Was that on this forum? What ever happened to him? That was indeed on this forum, and I have no idea what happened to him. Maybe he realised that all he'd invented was the Unix shebang? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Tue, 11 Oct 2022 at 14:26, wrote:
>
> I stand corrected Chris, and others, as I pay the sin tax.
>
> Yes, there are many kinds of errors that logically fall into different
> categories or phases of evaluation of a program and some can be determined
> by a more static analysis almost on a line by line (or "statement" or
> "expression", ...) basis and others need to sort of simulate some things
> and look back and forth to detect possible incompatibilities and yet others
> can only be detected at run time and likely way more categories depending on
> the language.
>
> But when I run the Python interpreter on code, aren't many such phases done
> interleaved and at once as various segments of code are parsed and examined
> and perhaps compiled into block code and eventually executed?
Hmm, depends what you mean. Broadly speaking, here's how it goes:
0) Early pre-parse steps that don't really matter to most programs,
like checking character set. We'll ignore these.
1) Tokenize the text of the program into a sequence of
potentially-meaningful units.
2) Parse those tokens into some sort of meaningful "sentence".
3) Compile the syntax tree into actual code.
4) Run that code.
Example:
>>> code = """def f():
... print("Hello, world", 1>=2)
... print(Ellipsis, ...)
... return True
... """
>>>
In step 1, all that happens is that a stream of characters (or bytes,
depending on your point of view) gets broken up into units.
>>> for t in tokenize.tokenize(iter(code.encode().split(b"\n")).__next__):
... print(tokenize.tok_name[t.exact_type], t.string)
It's pretty spammy, but you can see how the compiler sees the text.
Note that, at this stage, there's no real difference between the NAME
"def" and the NAME "print" - there are no language keywords yet.
Basically, all you're doing is figuring out punctuation and stuff.
Step 2 is what we'd normally consider "parsing". (It may well happen
concurrently and interleaved with tokenizing, and I'm giving a
simplified and conceptualized pipeline here, but this is broadly what
Python does.) This compares the stream of tokens to the grammar of a
Python program and attempts to figure out what it means. At this
point, the linear stream turns into a recursive syntax tree, but it's
still very abstract.
>>> import ast
>>> ast.dump(ast.parse(code))
"Module(body=[FunctionDef(name='f', args=arguments(posonlyargs=[],
args=[], kwonlyargs=[], kw_defaults=[], defaults=[]),
body=[Expr(value=Call(func=Name(id='print', ctx=Load()),
args=[Constant(value='Hello, world'), Compare(left=Constant(value=1),
ops=[GtE()], comparators=[Constant(value=2)])], keywords=[])),
Expr(value=Call(func=Name(id='print', ctx=Load()),
args=[Name(id='Ellipsis', ctx=Load()), Constant(value=Ellipsis)],
keywords=[])), Return(value=Constant(value=True))],
decorator_list=[])], type_ignores=[])"
(Side point: I would rather like to be able to
pprint.pprint(ast.parse(code)) but that isn't a thing, at least not
currently.)
This is where the vast majority of SyntaxErrors come from. Your code
is a sequence of tokens, but those tokens don't mean anything. It
doesn't make sense to say "print(def f[return)]" even though that'd
tokenize just fine. The trouble with the notion of "keeping going
after finding an error" is that, when you find an error, there are
almost always multiple possible ways that this COULD have been
interpreted differently. It's as likely to give nonsense results as
actually useful ones.
(Note that, in contrast to the tokenization stage, this version
distinguishes between the different types of word. The "def" has
resulted in a FunctionDef node, the "print" is a Name lookup, and both
"..." and "True" have now become Constant nodes - previously, "..."
was a special Ellipsis token, but "True" was just a NAME.)
Step 3: the abstract syntax tree gets parsed into actual runnable
code. This is where that small handful of other SyntaxErrors come
from. With these errors, you absolutely _could_ carry on and report
multiple; but it's not very likely that there'll actually *be* more
than one of them in a file. Here's some perfectly valid AST parsing:
>>> ast.dump(ast.parse("from __future__ import the_past"))
"Module(body=[ImportFrom(module='__future__',
names=[alias(name='the_past')], level=0)], type_ignores=[])"
>>> ast.dump(ast.parse("from __future__ import braces"))
"Module(body=[ImportFrom(module='__future__',
names=[alias(name='braces')], level=0)], type_ignores=[])"
>>> ast.dump(ast.parse("def f():\n\tdef g():\n\t\tnonlocal x\n"))
"Module(body=[FunctionDef(name='f', args=arguments(posonlyargs=[],
args=[], kwonlyargs=[], kw_defaults=[], defaults=[]),
body=[FunctionDef(name='g', args=arguments(posonlyargs=[], args=[],
kwonlyargs=[], kw_defaults=[], defaults=[]),
body=[Nonlocal(names=['x'])], decorator_list=[])],
decorator_list=[])], type_ignores=[])"
If you were to try to actually compile those to bytecode, they would fail:
>>> compile(ast.parse("from __future__ import braces"), "-", "exec")
Re: What to use for finding as many syntax errors as possible.
On Tue, 11 Oct 2022 at 14:13, wrote: > With the internet today, we are used to expecting error correction to come > for free. Do you really need one of every 8 bits to be a parity bit, which > only catches may half of the errors... Fortunately, we have WAY better schemes than simple parity, which was only really a thing in the modem days. (Though I would say that there's still a pretty clear distinction between a good message where everything has correct parity, and line noise where half of them don't.) Hamming codes can correct one-bit errors (and detect two-bit errors) at a price of log2(size)+1 bits of space. Here's a great rundown: https://www.youtube.com/watch?v=X8jsijhllIA There are other schemes too, but Hamming codes are beautifully elegant and easy to understand. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Tue, 11 Oct 2022 at 13:10, wrote: > If the above is: > > Import grumpy as np > > Then what happens if the code tries to find a file named "grumpy" somewhere > and cannot locate it and this is considered a syntax error rather than a > run-time error for whatever reason? Can you continue when all kinds of > functionality is missing and code asking to make a np.array([1,2,3]) clearly > fails? That's not a syntax error. Syntax is VERY specific. It is an error in Python to attempt to add 1 to "one", it is an error to attempt to look up the upper() method on None, it is an error to try to use a local variable you haven't assigned to yet, and it is an error to open a file that doesn't exist. But not one of these is a *syntax* error. Syntax errors are detected at the parsing stage, before any code gets run. The vast majority of syntax errors are grammar errors, where the code doesn't align with the parseable text of a Python program. (Non-grammatical parsing errors include using a "nonlocal" statement with a name that isn't found in any surrounding scope, using "await" in a non-async function, and attempting to import braces from the future.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Tue, 11 Oct 2022 at 09:18, Cameron Simpson wrote: > > On 11Oct2022 08:02, Chris Angelico wrote: > >There's a huge difference between non-fatal errors and syntactic > >errors. The OP wants the parser to magically skip over a fundamental > >syntactic error and still parse everything else correctly. That's > >never going to work perfectly, and the OP is surprised at this. > > The OP is not surprised by this, and explicitly expressed awareness that > resuming a parse had potential for "misparsing" further code. > > I remain of the opinion that one could resume a parse at the next > unindented line and get reasonable results a lot of the time. The next line at the same indentation level as the line with the error, or the next flush-left line? Either way, there's a weird and arbitrary gap before you start parsing again, and you still have no indication of what could make sense. Consider: if condition # no colon code else: code To actually "restart" parsing, you have to make a guess of some sort. Maybe you can figure out what the user meant to do, and parse accordingly; but if that's the case, keep going immediately, don't wait for an unindented line. If you want for a blank line followed by an unindented line, that might help with a notion of "next logical unit of code", but it's very much dependent on the coding style, and if you have a codebase that's so full of syntax errors that you actually want to see more than one, you probably don't have a codebase with pristine and beautiful code layout. > In fact, I expect that one could resume tokenising at almost any line > which didn't seem to be inside a string and often get reasonable > results. "Seem to be"? On what basis? > I grew up with C and Pascal compilers which would _happily_ produce many > complaints, usually accurate, and all manner of syntactic errors. They > didn't stop at the first syntax error. Yes, because they work with a much simpler grammar. But even then, most syntactic errors (again, this is not to be confused with semantic errors - if you say "char *x = 1.234;" then there's no parsing ambiguity but it's not going to compile) cause a fair degree of nonsense afterwards. The waters are a bit muddied by some things being called "syntax errors" when they're actually nothing at all to do with the parser. For instance: >>> def f(): ... await q ... File "", line 2 SyntaxError: 'await' outside async function This is not what I'm talking about; there's no parsing ambiguity here, and therefore no difficulty whatsoever in carrying on with the parsing. You could ast.parse() this code without an error. But resuming after a parsing error is fundamentally difficult, impossible without guesswork. > All you need in principle is a parser which goes "report syntax error > here, continue assuming ". For Python that might mean > "pretend a missing final colon" or "close open brackets" etc, depending > on the context. If you make conservative implied corrections you can get > a reasonable continued parse, enough to find further syntax errors. And, more likely, you'll generate a lot of nonsense. Take something like this: items = [ item[1], item2], item[3], ] As a human, you can easily see what the problem is. Try teaching a parser how to handle this. Most likely, you'll generate a spurious error - maybe the indentation, maybe the intended end of the list - but there's really only one error here. Reporting multiple errors isn't actually going to be at all helpful. > I remember the Pascal compiler in particular had a really good "you > missed a semicolon _back there_" mode which was almost always correct, a > nice boon when correcting mistakes. > Ahh yes. Design a language with strict syntactic requirements, and it's not too hard to find where the programmer has omitted them. Thing is Python just doesn't HAVE those semicolons. Let's say that a variant Python required you to put a U+251C ├ at the start of every statement, and U+2524 ┤ at the end of the statement. A whole lot of classes of error would be extremely easy to notice and correct, and thus you could resume parsing; but that isn't benefiting the programmer any. When you don't have that kind of information duplication, it's a lot harder to figure out how to cheat the fix and go back to parsing. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Tue, 11 Oct 2022 at 08:55, Robert Latest via Python-list wrote: > > Chris Angelico wrote: > > Yes, I'm aware that code readability becomes irrelevant for > > short-duration projects. Beside the point. I'm wondering how important > > it really is to have the shortest block first. > > I usually put the most expected / frequent / not negated block first if the > whole if/else statement is not "too long". Sometimes whatever you want to do > becomes pointless if a certain conditions is not met, in which case I do an > early break or return and have no else block at all. > > > Given that for-else is an excellent, if rarely-used, construct > > I knew it existed but coming from C I never thought to exploit it. I know I > wrote loops like this: > > found = None > while not found: > found = search(something) > if found: > break > if not found: > complain() > > Need to look into using "else" in these cases. Yep, that's exactly what for-else is great at! while True: if search(something): break else: complain() (although rather than a "while True", you're probably iterating over things to search for, in some way) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Tue, 11 Oct 2022 at 06:34, Peter J. Holzer wrote: > > On 2022-10-10 09:23:27 +1100, Chris Angelico wrote: > > On Mon, 10 Oct 2022 at 06:50, Antoon Pardon wrote: > > > I just want a parser that doesn't give up on encoutering the first syntax > > > error. Maybe do some semantic checking like checking the number of > > > parameters. > > > > That doesn't make sense though. > > I think you disagree with most compiler authors here. > > > It's one thing to keep going after finding a non-syntactic error, but > > an error of syntax *by definition* makes parsing the rest of the file > > dubious. > > Dubious but still useful. There's a huge difference between non-fatal errors and syntactic errors. The OP wants the parser to magically skip over a fundamental syntactic error and still parse everything else correctly. That's never going to work perfectly, and the OP is surprised at this. > > What would it even *mean* to not give up? > > Read the blog post on Lezer for some ideas: > https://marijnhaverbeke.nl/blog/lezer.html > > This is in the context of an editor. Incidentally, that's actually where I would expect to see that kind of feature show up the most - syntax highlighters will often be designed to "carry on, somehow" after a syntax error, even though it often won't make any sense (just look at what happens to your code highlighting when you omit a quote character). It still won't always be any use, but you do see *some* attempt at it. But if the OP would be satisfied with that, I rather doubt that this thread would even have happened. Unless, of course, the OP still lives in the dark ages when no text editor available had any suitable features for code highlighting. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 22:37, Axy via Python-list wrote: > > > On 10/10/2022 12:24, Chris Angelico wrote: > > On Mon, 10 Oct 2022 at 21:57, Axy via Python-list > > wrote: > >> > >>> Not sure what you mean, but a for-else without a break is quite > >>> useless. What exactly ARE you arguing here? > >>> > >>> The else is associated with the break to the exact extent that one is > >>> essential to the other's value. > >> I'm not arguing. That was just for the record, how things are done in > >> Python. Basically, I simply asked a question and got a definite answer > >> and clear understanding shortly, in a few replies. All the rest of this > >> thread looks irrelevant to me, it's about coding style and probably > >> should be continued under a different title, but I'm not interested to > >> participate in it. > > Here's where the "rest of this thread" started: > > > >> Actually the reason I never used "else" was the violation of the rule of > >> beauty "shortest block first". > > You disparaged a feature on the basis of a style rule that few of us > > had heard of or agree with. > Oh, I'm really sorry. My apologies. > > We all agree that coding style is > > important; none of us would see block length as a reason to avoid > > using an else clause on a for loop. > > As I understand from the above there must be a committee that delegates > a speaker? Where to read rules? How to participate? There's something > beyond this list I'm not aware of yet? We in this thread. Look at the past replies. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 21:57, Axy via Python-list wrote: > > > > Not sure what you mean, but a for-else without a break is quite > > useless. What exactly ARE you arguing here? > > > > The else is associated with the break to the exact extent that one is > > essential to the other's value. > > I'm not arguing. That was just for the record, how things are done in > Python. Basically, I simply asked a question and got a definite answer > and clear understanding shortly, in a few replies. All the rest of this > thread looks irrelevant to me, it's about coding style and probably > should be continued under a different title, but I'm not interested to > participate in it. Here's where the "rest of this thread" started: > Actually the reason I never used "else" was the violation of the rule of > beauty "shortest block first". You disparaged a feature on the basis of a style rule that few of us had heard of or agree with. We all agree that coding style is important; none of us would see block length as a reason to avoid using an else clause on a for loop. Your subsequent posts have left me confused as to what you're trying to convey. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 20:56, Axy via Python-list wrote: > > > The else is always coming with the break, not the for. > However, the compiler does not complain. > > There are [for ...], [for...break...], and[for...break...else], > > That's implied and contradicts Zen of Python, I think. If "else" came > with "break" there had to be a strong indication of that, namely > indentation, as it takes place for all other statements with their > clauses. However, there's no such an explicit connection between "break" > and "else". That's the point. > > Well, sorry for this addition to the discussion which went weird way. I > should had to be cautious mentioning particular coding style, that's a > totally different subject, actually. Let's close it at last. > > > but the [for...else] is insane. > Not in Python. > Not sure what you mean, but a for-else without a break is quite useless. What exactly ARE you arguing here? The else is associated with the break to the exact extent that one is essential to the other's value. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 20:46, Karsten Hilbert wrote: > > Am Sun, Oct 09, 2022 at 09:58:14AM + schrieb Stefan Ram: > > > I often follow this rule. For me, it's about readability. Compare: > > > > if not open( disk ): > > error( "Can't open disk" ) > > else: > > printf( "now imagine there's some larger block here" ) > ... ad infinitum > > Should this not be > > if not open( disk ): > error( "Can't open disk" ) > else: > do_lots_of_things_with(disk) > > as for readability ? > > Or even > > if not open( disk ): > error( "Can't open disk" ) > return what_needs_to_be_returned > > do_lots_of_things_with(disk) > > The latter version may need some code reorganization, though. I definitely prefer the fail-and-bail version (mainly because, as the number of possible failure modes increases, the code complexity increases linearly, whereas with if/else style it will increase quadratically - you have to edit the entire block, and indent it, for each one). But contrasting the original two? A complete wash. There's no readability difference between them. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 14:59, wrote: > > >>>Which is more disparaging: "I couldn't find anyone suggesting this" or > "The only place I could find it was a PHP style guide"? > >>>ChrisA > > Chris, > > If someone says they heard something from their own personal guru, people > often do not feel threatened or argue. I often am told nutrition or medical > or other advice that I simply ignore especially if it is about exotic herbs > to use or weird ideas like homeopathy or that I should use language X > because it is the only way to a haven or heaven or whatever. > > What we had here was someone suggesting their advice was WELL-KNOWN followed > by lots of people sputtering about not having heard of it. I actually think > the advice would be reasonable in many circumstances as long as it did not > conflict with dozens of goals I find more compelling but which vary on a > case by case basis such as whether I am prototyping something I will use > once, ... It's clearly not all that well known (none of us have heard of it, and it's not exactly prominent on the internet), and it seems that most of us disagree that it's even good advice. So, it's not really a good argument against for-else. > I have studied PHP but have never felt a need to use it and arguably the > roles it has played are often done commonly by other programs or methods. That's because PHP is terrible. > So in my view, the objection is not about PHP but about uniqueness. If the > author of one Python textbook and no others, suggest that your code should > declare all imports in alphabetical order then declare all functions in > alphabetical order, they can not only safely be ignored, but perhaps not > taken seriously as imports sometimes need to be done carefully if something > needs something else and if a language needs functions to be defined before > another function body calls them, ... It's not JUST about uniqueness. It's also that nobody but PHP programmers seem to care about it. That's on par with going to an art supplies forum and trying to argue that you should lay out your paints in a specific order, because some kindergarten teacher always does it that way for the kids' fingerpainting sessions. No, actually, that's unfair to fingerpainting kindergarteners. > But some people seem to miss a point we have discussed. The odd clauses like > ELSE after a loop (and quite a few variants in similar and different cases) > often provide guarantees such as making sure a file opened is closed. Citation needed. > Are these things out of the way? Yes, but so what? I see things as a whole > not as just a single screen shot. If a loop has several optional clauses lie > we are discussing and you know they are normally near the end, then look for > them where they are. So what's your point? > I know some languages, JavaScript being an example, may have things you > might consider odd such as promoting some things like function definitions > found further in your code to near the top so you can use a function that is > not defined till later even without something like a forward declaration > used in other languages. I've hardly ever seen good code that actually uses that. And when it did, it usually wasn't deliberate. Most well-written JS code will do the same thing that Python code does, calling things that have already been defined (if not lexically then temporally). No hoisting needed. > I am now going to stop replying on this topic as I have said way too much > and am not in particular disagreement if we are discussing preferences and > ideas. I see TOOLS here, not religion. Use what works or that has to be used > for a task but don't take any one thing too seriously. Yes, I see tools too. And this thread started out with a discussion of the for-else construct, which was disparaged because it violated a rule that nobody here has heard of, few agree with, and has exceptions already. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 11:52, MRAB wrote: > > On 2022-10-10 00:40, dn wrote: > > On Sun, 9 Oct 2022 at 15:39, Axy via Python-list > > wrote: > > > >> "shortest block first" > > > > Have never heard this advice before. Kind-of rankled with me, as it did > > for others. > > > > Enquiring minds want to know... Played Duck, duck, go on this: zero hits > > amongst a pile of similar phrases - turns-out there's an algorithm with > > a similar name, but not related, and an electronics approach (way too > > 'low' a level for translation to us though). > > > > Tried prefixing with "program" but no such advice to programmers or > > program[me] designers. > > > > Tried prefixing with "python", but equal lack of joy. > > > > Would OP please quote source? > > > [snip] > After a few minutes searching I found this: > > https://docs.typo3.org/m/typo3/reference-coreapi/9.5/en-us/CodingGuidelines/CglPhp/PhpFileFormatting/PhpSyntaxFormatting.html > > """It is recommended to create conditions so that the shortest block of > code goes first.""" Which is more disparaging: "I couldn't find anyone suggesting this" or "The only place I could find it was a PHP style guide"? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What to use for finding as many syntax errors as possible.
On Mon, 10 Oct 2022 at 06:50, Antoon Pardon wrote: > I just want a parser that doesn't give up on encoutering the first syntax > error. Maybe do some semantic checking like checking the number of parameters. That doesn't make sense though. It's one thing to keep going after finding a non-syntactic error, but an error of syntax *by definition* makes parsing the rest of the file dubious. What would it even *mean* to not give up? How should it interpret the following lines of code? All it can do is report the error. You know, if you'd not made this thread, the time you saved would have been enough for quite a few iterations of "fix one syntactic error, run it again to find the next". ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 03:46, Avi Gross wrote: > > Chris, I was not arguing that at all. Maybe not intentionally, but you did lend a lot of weight to that argument :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Mon, 10 Oct 2022 at 03:22, Avi Gross wrote: > > Smallest code blocks first may be a more modern invention. > > Some would argue for a rule related to efficiency of execution. When you > have multiple blocks as in an if-else or case statement with multiple > choices, that you order the most common cases first. Those shorten > execution more often than the rarer cases especially the ones that should > never happen. > Seems fairly dubious and full of special-cases. If you want to follow that rule, it should be easy enough to still permit for-else clauses. It's an extremely weak argument against for-else. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Sun, 9 Oct 2022 at 16:05, Axy via Python-list wrote: > > > On 09/10/2022 05:47, Chris Angelico wrote: > > On Sun, 9 Oct 2022 at 15:39, Axy via Python-list > > wrote: > >> Got it, thanks! > >> > >> Actually the reason I never used "else" was the violation of the rule of > >> beauty "shortest block first". With if--else you can easily follow this > >> rule by inverting "if" expression, but with for--else you can't. The > >> loop body of the simplest example is already three lines, in real life > >> things are much worse. > >> > > That's not a rule I've ever been taught; how important is it? > > > > ChrisA > > It gets important if the lifetime of your project is more than three > months and is extremely important if more than 10 years. But, it depends. Yes, I'm aware that code readability becomes irrelevant for short-duration projects. Beside the point. I'm wondering how important it really is to have the shortest block first. > I also might be wrong in terminology, anyway, there are many rules that > make programmer's life easier, described in the literature from the old > good "How to write unmaintainable code" to "The Art of Readable Code". > And I hope there are a lot of recent books on this subject I did not > track and read yet. Also not really a justification for "shortest block first". Wanting some elaboration on that. What's the value in it? Given that for-else is an excellent, if rarely-used, construct, I would say that, *at least*, it is worth setting aside this rule for that particular situation. It is also generally worth using fewer commas than I just did. Take my advice with a grain of salt. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: for -- else: what was the motivation?
On Sun, 9 Oct 2022 at 15:39, Axy via Python-list wrote: > > Got it, thanks! > > Actually the reason I never used "else" was the violation of the rule of > beauty "shortest block first". With if--else you can easily follow this > rule by inverting "if" expression, but with for--else you can't. The > loop body of the simplest example is already three lines, in real life > things are much worse. > That's not a rule I've ever been taught; how important is it? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Debugging automatic quotation in subprocess.Popen()
On Sat, 8 Oct 2022 at 08:24, wrote: > > Hello, > > I need to improve my understanding about how subprocess.Popen() does > quote arguments. I have special case here. > > Simple example: > Popen(['ls', '-l']) results on a shell in "ls -l" without quotation. > > Quotes are added if they are needed: > Popen(['ls', 'folder with blank']) results on a shell in > "ls 'folder with blank'". > > Am I right so far with the basics? > > Is there a way to be sure and to debug how Popen() give it to the shell? That's kinda looking at it backwards; the shell first splits the command into a list of arguments, and then runs it. Python has a module that is capable of doing similar sorts of work: https://docs.python.org/3/library/shlex.html That may be helpful if you want to give the user something to copy and paste. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Implementation of an lru_cache() decorator that ignores the first argument
On Thu, 29 Sept 2022 at 05:36, Robert Latest via Python-list wrote: > in a (Flask) web application I often find that many equal (SQLAlchemy) queries > are executed across subsequent requests. So I tried to cache the results of > those queries on the module level like this: > > @lru_cache() > def query_db(db, args): > # do the "expensive" query > return result > > ... > This is what I came up with. I'm quite happy with it so far. Question: Am I > being too clever? is it too complicated? Am I overlooking something that will > come back and bite me later? Thanks for any comments! > > def lru_ignore_first(timeout=0, **lru_args): > ... I think this code is fairly specific to what you're doing, which means the decorator won't be as reusable (first hint of that is the entire "timeout" feature, which isn't mentioned at all in the function's name). So it's probably not worth trying to do this multi-layered approach, and it would be as effective, and a lot simpler, to just have code at the top of the query_db function to do the cache lookup. But you may find that your database is *itself* able to do this caching for you, and it will know when to evict from cache. If you really have to do it yourself, keep it really really simple, but have an easy way *in your own code* to do the cache purge; that way, you guarantee correctness, even at the expense of some performance. In terms of overall database performance, though: are you using transactions correctly? With PostgreSQL, especially, the cost of doing a series of queries in one transaction is barely higher than doing a single query in a transaction; or, putting it the other way around, doing several sequential transactions costs several times as much as doing one combined transaction. Check to see that you aren't accidentally running in autocommit mode or anything. It could save you a lot of hassle! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: on implementing a toy oop-system
On Sat, 24 Sept 2022 at 07:52, Meredith Montgomery
wrote:
>
> def Counter(name = None):
> o = {"name": name if name else "untitled", "n": 0}
> def inc(o):
> o["n"] += 1
> return o
> o["inc"] = inc
> def get(o):
> return o["n"]
> o["get"] = get
> return o
>
Want a neat demo of how classes and closures are practically the same thing?
def Counter(name=None):
if not name: name = "untitled"
n = 0
def inc():
nonlocal n; n += 1
def get():
return n
return locals()
Aside from using a nonlocal declaration rather than "self.n", this is
extremely similar to classes, yet there are no classes involved.
A class statement creates a namespace. The locals() function returns
the function's namespace.
Each call to Counter() creates a new closure context, just like each
call to a constructor creates a new object.
There's very little difference, at a fundamental level :)
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to get the current set LOG_MASK in Python's syslog module?
On Thu, 22 Sept 2022 at 23:46, Richard Moseley wrote: > > According to documentation syslog.setlogmask returns the current mask so > save the value to reset later on. > > Oldval = syslog.setlogmask(newmask) > > This sets oldval to original mask. This on its own suggests an odd technique that should work but won't be great: oldval = syslog.setlogmask(1234) syslog.setlogmask(oldval) But the Python function just passes the value straight to the underlying system call, and thus treats zero specially: """ The setlogmask() function sets this logmask for the calling process, and returns the previous mask. If the mask argument is 0, the current logmask is not modified. """ So you should be able to do: current_mask = syslog.setlogmask(0) The Python docs do say to refer to the man pages, but IMO it would be worth mentioning this feature specifically in the docs. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How to replace an instance method?
On Sun, 18 Sept 2022 at 10:29, wrote: > > > From your description, Chris, it sounds like the functional programming > technique often called currying. A factory function is created where one (or > more) parameters are sort of frozen in so the user never sees or cares about > them, and a modified or wrapped function is returned. In this case, the > function now knows what object it is attached to as this. > Correct. I avoided the technical term but that is exactly what's happening. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How to replace an instance method?
On Sun, 18 Sept 2022 at 09:37, Eryk Sun wrote: > > On 9/17/22, Chris Angelico wrote: > > > > The two are basically equivalent. Using functools.partial emphasizes > > the fact that all you're doing is "locking in" the first parameter; > > using the __get__ method emphasizes the fact that functions are, > > fundamentally, the same thing as methods. Choose whichever one makes > > sense to you! > > Functions are really not "fundamentally, the same thing as methods". > They're only the same in that they're both callable. Also, a method's > __getattribute__() falls back on looking up attributes on the > underlying function (i.e. the method's __func__), such as inspecting > the __name__ and __code__. A fundamental difference is that, unlike a > function, a method is not a descriptor. Thus if a method object is set > as an attribute of a type, the method does not rebind as a new method > when accessed as an attribute of an instance of the type. An unbound method in Python 2 was distinctly different from a function, but in Python 3, they really truly are the same thing. A bound method object is a small wrapper around a function which binds its 'self' parameter; that's a distinction, but not a fundamental one. Yes, a bound method isn't a descriptor; that's not really a huge difference either, though. A method IS a function. A bound method is a function with one argument locked in, but still a function. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How to replace an instance method?
On Sun, 18 Sept 2022 at 07:20, Ralf M. wrote: > > Am 16.09.2022 um 23:34 schrieb Eryk Sun: > > On 9/16/22, Ralf M. wrote: > >> I would like to replace a method of an instance, but don't know how to > >> do it properly. > > > > A function is a descriptor that binds to any object as a method. For > > example: > > > > >>> f = lambda self, x: self + x > > >>> o = 42 > > >>> m = f.__get__(o) > > >>> type(m) > > > > >>> m.__self__ is o > > True > > >>> m(10) > > 52 > > Thank you and Chris A. for the two suggestions how to replace a method. > > I tried both >inst.method = functools.partial(new_method, inst) > and >inst.method = new_method.__get__(inst) > and both work in my toy example. > I will try it on the real code next week. > > Even though the functools.partial solution is easier to understand (at > least for me), I will probably use the __get__ solution as it avoids > the import of an extra library. > The two are basically equivalent. Using functools.partial emphasizes the fact that all you're doing is "locking in" the first parameter; using the __get__ method emphasizes the fact that functions are, fundamentally, the same thing as methods. Choose whichever one makes sense to you! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: How to replace an instance method?
On Sat, 17 Sept 2022 at 07:07, Ralf M. wrote: > > I would like to replace a method of an instance, but don't know how to > do it properly. > > My first naive idea was > > inst = SomeClass() > def new_method(self, param): > # do something > return whatever > inst.method = new_method > > however that doesn't work: self isn't passed as first parameter to > the new inst.method, instead inst.method behaves like a static method. > > I had a closer look at the decorators classmethod and staticmethod. > Unfortunetely I couldn't find a decorator / function "instancemethod" > that turns a normal function into an instancemethod. > > The classmethod documentation contains a reference to the standard > type hierarchie, and there is an explanation that an instancemethod > is sort of a dynamically created wrapper around a function, which > is accessable as __func__. > So I modified the last line of the example above to > > inst.method.__func__ = new_method > > but got told that __func__ is read only. > > I found some information about methods in the Descriptor HowTo Guide, > but it's about how it works internally and doesn't tell how to solve > my problem (at least it doesn't tell me). > > Now I'm running out of ideas what to try next or what sections of the > documentation to read next. > > Any ideas / pointers? > You don't actually want a descriptor, since the purpose of descriptor protocol is to give you information about the instance when the attribute (the method, in this case) was attached to the class. In this case, you can handle it with something far far simpler: inst.method = functools.partial(new_method, inst) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about learning Python
On Sat, 10 Sept 2022 at 06:45, Dennis Lee Bieber wrote: > > On Thu, 8 Sep 2022 07:42:19 +1200, dn > declaimed the following: > > >TSRs? Now that was an ugly period of history! (trying to make a > >single-process operating system do multi-processing - only to find that > >many program[me]s assumed they had full use and undisputed control of > >the computer. Happy days...) > > > > I laughed when M$ MSDOS (2?) introduced TSRs... My TRS-80 running > L(S)DOS had similar things at least a year earlier. And these were > /run-time/ loadable. They called them "filters" (and device drivers were > also an option). Key-click was one such -- though it also showed some > quirks (like... If the processor was really busy, the key-board driver > would buffer key-strokes, but the filter activated when an application > /read/ the key-board). Filter to control printer formatting, a JobLog > filter, Key-Stroke Multiply filter (I never used it, but it apparently uses > a table of special keys and expands them to longer strings). Commands to > load device drivers (or remove them!). Could even change the number of > cylinders for a floppy drive -- My drives were "loose" enough to allow my > to add 2 cylinders. > To be fair on MS-DOS, you didn't *have* to use a TSR to hook interrupts, and the same idea of those filters would work (just hook the keyboard interrupt in a cooperative way; last installed is first executed). But the OS offered only one option for a program to run and put itself somewhere: "terminate process and increase the base segment address for subsequent processes", which would allow you to leave any amount of memory (on a sixteen-byte boundary) active. There's no reason that filters couldn't have been written that blit themselves into some other part of memory, point some interrupt vectors there, and then fully terminate. Sure, the OS wouldn't have offered any protection, but the OS didn't offer any actual protection anyway. All we'd need is a spare slab of memory for things to put their code into, one which everyone could allocate and deallocate from. oh. Yeah, like virtual memory. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about learning Python
On Sat, 10 Sept 2022 at 06:38, Greg Ewing wrote: > > On 8/09/22 6:57 am, Chris Angelico wrote: > > Not as detrimental as starting with BASIC, and then moving on to x86 > > assembly language, and trying to massage the two together using CALL > > ABSOLUTE in order to get mouse input in your GW-BASIC programs. > > Or starting with hand-assembled SC/MP machine code and then moving > on to Applesoft BASIC. > I have no idea how we survived. Though, "survived with our sanity intact" clearly didn't happen, so perhaps there's that. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about learning Python
On Thu, 8 Sept 2022 at 05:09, Grant Edwards wrote: > > On 2022-09-07, Chris Angelico wrote: > > On Thu, 8 Sept 2022 at 04:54, Grant Edwards > > wrote: > > > >> If you're a beginning programmer, then IMO learning C first is > >> probably detrimental. [...] > > > > Not as detrimental as starting with BASIC, and then moving on to x86 > > assembly language, and trying to massage the two together using CALL > > ABSOLUTE in order to get mouse input in your GW-BASIC programs. > > Ah the "good old days". > Indeed. The 1990s gave me all manner of skills, including the aforementioned mouse control in a BASIC program, writing a Terminate-and-Stay-Resident program that hooks an interrupt, tricks for *not* writing a TSR and still acting like one, building GUIs using pixel precision, building GUIs using pixel precision but fully automatically, using HTML tables to create layouts oh, yes, so many skills... To anyone suffering from https://xkcd.com/1479/ right now, I can assure you, quite a lot of that knowledge DOES eventually become obsolete when better methods come along. It just sometimes takes a decade or more. (And then occasionally it still haunts you. I'm finding table-based layouts in a site that I now have to manage. Eventually I'll fix it all, eventually) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about learning Python
On Thu, 8 Sept 2022 at 04:54, Grant Edwards wrote: > > On 2022-09-07, Chris Angelico wrote: > > On Thu, 8 Sept 2022 at 01:50, Maruful Islam wrote: > >> > >> I want to start learning python. I have a question about learning python. > >> > >> Is learning C essential or not for learning python? > > > > Absolutely not essential. In fact, I would strongly recommend learning > > Python before ever picking up C, as it's much easier to mess around in > > Python. > > If you're a beginning programmer, then IMO learning C first is > probably detrimental. C has quite a few quirks and pitfalls that will > a) frustrate you and waste time, and b) influence your way of thinking > about programs in a way that will be unhelpful for higher level > languages. Not as detrimental as starting with BASIC, and then moving on to x86 assembly language, and trying to massage the two together using CALL ABSOLUTE in order to get mouse input in your GW-BASIC programs. Don't be me, folks. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Question about learning Python
On Thu, 8 Sept 2022 at 01:50, Maruful Islam wrote: > > I want to start learning python. I have a question about learning python. > > Is learning C essential or not for learning python? Absolutely not essential. In fact, I would strongly recommend learning Python before ever picking up C, as it's much easier to mess around in Python. Learning C will definitely help you to become a better programmer, but you can leave it until later. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
