Re: absolute path to a file
Hi Paul, Here an example how I used both functions https://gitlab.com/snippets/1886520 Hope this helps. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: absolute path to a file
On Thu, 15 Aug 2019 22:00:17 +0200
Paul St George wrote:
> Can someone please tell me how to get the absolute path to a file? I
> have tried os.path.abspath. In the code below I have a problem in the
> final line (15).
>
> #
> |import bpy||
> ||import os||
> ||
> ||texture_list = []||
> ||
> ||with open(os.path.splitext(bpy.data.filepath)[0] + ".txt", "w") as
> outstream:||
> ||
> ||
> || for obj in bpy.context.scene.objects:||
> || for s in obj.material_slots:||
> || if s.material and s.material.use_nodes:||
> || for n in s.material.node_tree.nodes:||
> || if n.type == 'TEX_IMAGE':||
> || texture_list += [n.image]||
> ||print(obj.name,'uses',n.image.name,'saved at',n.image.filepath,
> 'which is at', os.path.abspath(n.image.filepath), file=outstream)||
> |#
>
> This gives me:
> ---Plane uses image01.tif saved at //image01.tif which is
> at //image01.tif ---Plane uses image02.tif saved
> at //../images/image02.tif which is at //images/image02.tif
>
> But I want an absolute path such as:
> ---Plane uses image01.tif saved
> at /Users/Lion/Desktop/test8/image01.tif ---Plane uses image02.tif
> saved at /Users/Lion/Desktop/images/image02.tif
>
> If it is relevant, my files are on a Mac. Hence the escaped forward
> slash.
>
I did this:
from pathlib import Path
abs_myfile = Path('./myfile').resolve()
which worked fine for me.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: absolute path to a file
On Fri, 16 Aug 2019 09:00:38 +1000
Cameron Simpson wrote:
> On 15Aug2019 22:52, Manfred Lotz wrote:
> >I did this:
> >from pathlib import Path
> >abs_myfile = Path('./myfile').resolve()
> >which worked fine for me.
>
> There is also os.path.realpath(filename) for this purpose. In modern
> Python that also accepts a Pathlike object.
Thanks for this.
I changed my code to use your suggestion which seems to
be better for the situation where I used resolve() before.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
https://test.pypi.org/ internal server error
I want to exercise how to create a package and upload it to pyi. So, I registered with https://test.pypi.org/. Now, when I click on https://test.pypi.org/manage/account/#account-emails to verify my email address I get an internal server error. Same when I try https://test.pypi.org/manage/account/. What is the proper place to report the problem? -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: https://test.pypi.org/ internal server error
On Sat, 24 Aug 2019 14:12:38 +0200 Manfred Lotz wrote: > I want to exercise how to create a package and upload it to pyi. So, I > registered with https://test.pypi.org/. > > Now, when I click on > https://test.pypi.org/manage/account/#account-emails to verify my > email address I get an internal server error. > > Same when I try https://test.pypi.org/manage/account/. > > What is the proper place to report the problem? > For the records. In the meantime the internal server error has been fixed. -- https://mail.python.org/mailman/listinfo/python-list
Re: open, close
On Sat, 31 Aug 2019 16:37:23 +0200
Peter Otten <__pete...@web.de> wrote:
> Manfred Lotz wrote:
>
> > Hi there,
> > This is a beginner question.
> >
> > I learned that
> >
> > with open("foo.txt") as f:
> > lines = f.readlines()
> >
> > using the with-construct is the recommended way to deal with files
> > making sure that close() always happens.
> >
> > However, I also could do:
> >
> > lines = open("foo.txt").readlines()
> >
> > I have to admit that I'm not sure if in case something bad happens a
> > close() is done implicitly as in the first example.
> >
> >
> > Could I use the latter as a substitute for the with-construct? What
> > are the recommendations of the experts?
>
> Always using
>
> with open(...) ...
>
> is a good habit to get into.
As a Python beginner I started to use with open.. in all cases but was
tempted to use the even shorter one-line way of reading a file.
So, I will continue to use with open... ALWAYS as good habits are very
helpful in life. :-)
> if you need to read all lines of a file
> very often write a helper function:
>
> def readlines(filename):
> with open(filename) as f:
> return f.readlines()
>
> That way you can write
>
> lines = readlines("foo.txt")
>
> which saves even more typing and still closes the file
> deterministically.
>
Good idea.
Thanks to all for your help.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
open, close
Hi there,
This is a beginner question.
I learned that
with open("foo.txt") as f:
lines = f.readlines()
using the with-construct is the recommended way to deal with files
making sure that close() always happens.
However, I also could do:
lines = open("foo.txt").readlines()
I have to admit that I'm not sure if in case something bad happens a
close() is done implicitly as in the first example.
Could I use the latter as a substitute for the with-construct? What are
the recommendations of the experts?
--
Thanks, Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: open, close
On Sat, 31 Aug 2019 15:43:41 +0200
Piet van Oostrum wrote:
> Max Zettlmeißl writes:
>
> > On Sat, Aug 31, 2019 at 2:22 PM Manfred Lotz
> > wrote:
> >>
> >> Could I use the latter as a substitute for the with-construct?
> >>
> >
> > You can't use the second statement as a proper substitute for the
> > first one.
> >
> > With the context manager, it is ensured that the file is closed.
> > It's more or less equal to a "finally" clause which closes the file
> > descriptor.
> > So as long as the Python runtime environment functions properly, it
> > will be closed.
> >
> > Your second statement on the other hand, is more or less equivalent
> > to:
> >
> > f = open("foo.txt")
> > lines = f.readlines()
> >
> > Close won't be called.
>
> There is a difference here with the construct that the OP mentioned:
>
> lines = open("foo.txt").readlines()
>
> In that case the file COULD be closed, but there is no guarantee. It
> depends on garbage collection. In your case the file will not be
> closed as long as there is still a reference to it (as in f). When f
> disappears and all copies of it as well, the file COULD be closed
> similarly.
>
When you say COULD this sounds like it is a matter of luck. My thinking
was that USUALLY the file will be closed after the statement because
then the file handle goes out of scope.
However, I understand now that in contrary to `with...` there is no
guarantee.
> On the other hand, the with statement guarantees that the file will
> be closed, so it is the preferred method.
>
Thanks. So, I take that I'm better off to use 'with...'.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: open, close
On Sun, 1 Sep 2019 16:46:44 +0100 Barry wrote: > > On 31 Aug 2019, at 15:41, Manfred Lotz wrote: > > > > When you say COULD this sounds like it is a matter of luck. My > > thinking was that USUALLY the file will be closed after the > > statement because then the file handle goes out of scope. > > It all depends on the way any python implementation does its garbage > collection. The file is closed as a side effect of deleting the file > object to reclaiming the memory of the file object. > > At the start of python 3 people where suprised when files and other > resources where not released at the same time that python 2 released > them. > Thanks for this. Very interesting. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to record the console's content in the interpreter?
On Thu, 22 Aug 2019 21:48:33 -0700 (PDT) jf...@ms4.hinet.net wrote: > Cameron Simpson於 2019年8月23日星期五 UTC+8下午12時09分54秒寫道： > > On 22Aug2019 19:38, Jach Fong wrote: > > >Say I like to record everything showing in the console into a file > > >after I start a debug session, and stop it when finished. It's not > > >a console redirection. I still need to see what is going on during > > >the session. > > > > If you're in a terminal you can run the 'script" command. This > > starts a new shell inside a session which records itself to the > > file "typescript" (by default). See "man script" for further > > details. > > > > Cheers, > > Cameron Simpson > > Mine is Windows:-( No 'script' command? > > --Jach If I understand you correctly you want to save an interactive session!? Then you could use ipython. In ipython the lines get numbererd. If your last line is, for example: In [100] you could do this: save mysave 1-100 in the ipython shell. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to record the console's content in the interpreter?
On Fri, 23 Aug 2019 00:42:26 -0700 (PDT) jf...@ms4.hinet.net wrote: > Manfred Lotz於 2019年8月23日星期五 UTC+8下午2時58分48秒寫道： > > On Thu, 22 Aug 2019 21:48:33 -0700 (PDT) > > jf...@ms4.hinet.net wrote: > > > > > Cameron Simpson於 2019年8月23日星期五 > > > UTC+8下午12時09分54秒寫道： > > > > On 22Aug2019 19:38, Jach Fong wrote: > > > > >Say I like to record everything showing in the console into a > > > > >file after I start a debug session, and stop it when finished. > > > > >It's not a console redirection. I still need to see what is > > > > >going on during the session. > > > > > > > > If you're in a terminal you can run the 'script" command. This > > > > starts a new shell inside a session which records itself to the > > > > file "typescript" (by default). See "man script" for further > > > > details. > > > > > > > > Cheers, > > > > Cameron Simpson > > > > > > Mine is Windows:-( No 'script' command? > > > > > > --Jach > > > > If I understand you correctly you want to save an interactive > > session!? > > > > Then you could use ipython. In ipython the lines get numbererd. If > > your last line is, for example: In [100] you could do this: > > > > save mysave 1-100 > > > > in the ipython shell. > > > > -- > > Manfred > > After a quick try on IPython 6.5.0, I saw only the commands I had > typed, not include the output of those commands:-( > Sorry, that's true. I found only this. After starting ipython type: %logstart -o Then you get also command output. The log file name is: ipython_log.py But it doesn't seem to show output from print command. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Spread a statement over various lines
I have a function like follows
def regex_from_filepat(fpat):
rfpat = fpat.replace('.', '\\.') \
.replace('%', '.') \
.replace('*', '.*')
return '^' + rfpat + '$'
As I don't want to have the replace() functions in one line my
question is if it is ok to spread the statement over various lines as
shown above, or if there is a better way?
Thanks.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: Spread a statement over various lines
On Tue, 17 Sep 2019 20:59:47 +0200
Manfred Lotz wrote:
> I have a function like follows
>
> def regex_from_filepat(fpat):
> rfpat = fpat.replace('.', '\\.') \
> .replace('%', '.') \
Not related to my question but the second replace must be:
.replace('?', '.')
--
https://mail.python.org/mailman/listinfo/python-list
Re: How do I purge pip intsall --user packages?
On 17 Sep 2019 19:10:29 GMT Martin Schöön wrote: > I have installed a bunch of packages using pip install --user and > I went for a non-standard location for the install. Now I regret > this and would like to wipe this and start all over again using > the standard location. Is it enough to delete the folder I > specified or am I missing something? Having to uninstall each > and every package would be tedious... > > /Martin Do you want to uninstall all per user installed packages? Then you could try: pip3 list --user | tail -n +3 | while read p rest; do pip3 uninstall $p;done -- https://mail.python.org/mailman/listinfo/python-list
Re: Spread a statement over various lines
On Tue, 17 Sep 2019 16:51:04 -0400
Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> wrote:
> On 9/17/19 2:59 PM, Manfred Lotz wrote:
>
> > def regex_from_filepat(fpat):
> > rfpat = fpat.replace('.', '\\.') \
> >.replace('%', '.') \
> >.replace('*', '.*')
> >
> > return '^' + rfpat + '$'
> >
> > As I don't want to have the replace() functions in one line my
> > question is if it is ok to spread the statement over various lines
> > as shown above, or if there is a better way?
>
> Is that way okay? Sure. Are there other ways? Sure.
>
> To isolate each replace() function on its own line, and to eliminate
> the clutter of the backslashes, consider this:
>
> rfpat = (fpat
>.replace('.', '\\.')
>.replace('%', '.')
>.replace('*', '.*')
> )
>
> "Better" is going to depend on your initial reason for not wanting all
> those function calls on one line. What is that reason?
Well, I don't want those functions on a single line because having them
multiline the code looks clearer.
I asked here because I don't like the backslashes. Wrapping them in ()
looks like a good solution.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Spread a statement over various lines
On Wed, 18 Sep 2019 09:52:21 +0200
Wolfgang Maier wrote:
> On 17.09.19 20:59, Manfred Lotz wrote:
> > I have a function like follows
> >
> > def regex_from_filepat(fpat):
> > rfpat = fpat.replace('.', '\\.') \
> > .replace('%', '.') \
> > .replace('*', '.*')
> >
> > return '^' + rfpat + '$'
> >
> >
> > As I don't want to have the replace() functions in one line my
> > question is if it is ok to spread the statement over various lines
> > as shown above, or if there is a better way?
> >
>
> One problem with explicit line continuation using \ is that it is
> dependent on the backslash being the last character on the line, i.e.
> a single space at the end of the line will result in a SyntaxError.
> This is why implicit line continuation relying on opened parentheses,
> brackets or curly braces is often preferred, and recommended over
> backslash continuation in PEP8
> (https://www.python.org/dev/peps/pep-0008/#maximum-line-length).
> To use implicit line continuation you could either introduce extra
> surrounding parentheses as suggested by others, or you may make use
> of the parentheses you have already, like so:
>
> def regex_from_filepat(fpat):
> rfpat = fpat.replace(
> '.', '\\.'
> ).replace(
> '%', '.'
> ).replace(
> '*', '.*'
> )
>
> return '^' + rfpat + '$'
>
>
Thanks for showing. Good to be aware of this possiblity. So, it seems
it doesn't hurt to read PEP8.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Spread a statement over various lines
On Wed, 18 Sep 2019 08:30:08 +0200
Peter Otten <__pete...@web.de> wrote:
> Manfred Lotz wrote:
>
> > I have a function like follows
> >
> > def regex_from_filepat(fpat):
> > rfpat = fpat.replace('.', '\\.') \
> > .replace('%', '.') \
> > .replace('*', '.*')
> >
> > return '^' + rfpat + '$'
> >
> >
> > As I don't want to have the replace() functions in one line my
> > question is if it is ok to spread the statement over various lines
> > as shown above, or if there is a better way?
>
> Sometimes you can avoid method-chaining:
>
> >>> REP = str.maketrans({".": "\\.", "%": ".", "*": ".*"})
> >>> def regex_from_filepat(fpat):
> ... return fpat.translate(REP)
> ...
> >>> regex_from_filepat("*foo.%")
> '.*foo\\..'
>
Very interesting. Thanks for this.
> Generally speaking a long statement may be a hint that there is an
> alternative spelling.
>
>
I'll keep it in my mind.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Spread a statement over various lines
On Thu, 19 Sep 2019 08:36:04 +0200 Peter Otten <__pete...@web.de> wrote: > Manfred Lotz wrote: > > >> Not related to your question, but: > >> You seem to try to convert a Windows wildcard pattern to a regex > >> pattern. > > > > No, I'm on Linux. > > > > Shortly, after I had posted the question I discovered fnmatch() in > > the standard library, and I changed my code accordingly. > > I would have pointed to fnmatch, but I didn't recognize the '%' that > you used instead of the usual '?', and fnmatch doesn't either, I > think. > > Where does '%' come from? > '%' was a mistake as I had replied myself to my initial question. -- https://mail.python.org/mailman/listinfo/python-list
Re: Spread a statement over various lines
On Wed, 18 Sep 2019 22:01:34 +0200
"Ralf M." wrote:
> Am 17.09.2019 um 20:59 schrieb Manfred Lotz:
> > I have a function like follows
> >
> > def regex_from_filepat(fpat):
> > rfpat = fpat.replace('.', '\\.') \
> >.replace('%', '.') \
> >.replace('*', '.*')
> >
> > return '^' + rfpat + '$'
> >
> >
> > As I don't want to have the replace() functions in one line my
> > question is if it is ok to spread the statement over various lines
> > as shown above, or if there is a better way?
> >
> > Thanks.
> >
>
> Not related to your question, but:
> You seem to try to convert a Windows wildcard pattern to a regex
> pattern.
No, I'm on Linux.
Shortly, after I had posted the question I discovered fnmatch() in the
standard library, and I changed my code accordingly.
Nevertheless, I was happy to have asked the question as the
following discussion was very interesting for me.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
class definition question
Hi there, More often I see something like this: class Myclass: ... but sometimes I see class Myclass(object): ... Question: which way is preferable? -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: class definition question
On Wed, 07 Aug 2019 14:39:00 -0400 Dennis Lee Bieber wrote: > On Wed, 7 Aug 2019 20:11:15 +0200, Manfred Lotz > declaimed the following: > > >Hi there, > >More often I see something like this: > > > >class Myclass: > >... > > > > > >but sometimes I see > > > >class Myclass(object): > >... > > > > > >Question: which way is preferable? > > It's historical... > > Python v1.x had a form of classes. > > Python v2.x introduced "new-style" classes. "New-style" > classes /had/ to inherit from "object", as they had different > behavior from "v1.x old-style" classes which were still supported (it > would have broken too many programs). Old-style were then deprecated, > and one should have used new-style for new code. > > Python v3.x unified (removed old-style behavior differences) > and all classes inherit from "object" whether one specifies object or > not. > > Thanks a lot for the explanations. As a Python newbie (with no Pythons legacies) I only deal with Python 3. So, I will happily ignore 'object'. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Proper shebang for python3
On Sun, 21 Jul 2019 03:44:24 +1000 Chris Angelico wrote: > On Sun, Jul 21, 2019 at 3:41 AM Manfred Lotz > wrote: > > > > Hi there, > > Pretty new to python I've got a question regarding the proper > > shebang for Python 3. > > > > I use > >#!/usr/bin/python3 > > > > which works fine. > > > > Today I saw > >#!/usr/bin/python3 -tt > > > > and was wondering what -tt means. > > > > Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't > > even mention -t. > > > > python 2 man page mentions > > > >-t Issue a warning when a source file mixes tabs and > > spaces for indentation in a way that makes it depend on the > > worth of a tab expressed in spaces. Issue an error when the option > > is given twice. > > > > I guess that -t has the same meaning with python 3.7.3. > > Far as I know, the -tt option to Python 3 is accepted for 2/3 > compatibility, but does absolutely nothing. Having it on the shebang > probably means that it used to be /usr/bin/python -tt, and got > migrated to python3 without removing it. > This is a plausible explanation. Thanks, Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Proper shebang for python3
On Sat, 20 Jul 2019 14:11:21 -0400 Michael Speer wrote: > You may want to use `#!/usr/bin/env python3` instead. > In my case it doesn't matter. However, I agree that your suggestion is usually preferable. > There is a concept in python called the virtual environment. This > used to be done with a tool called virtualenv in python2, and is now > done mainly through a venv module in python3. > > A virtual environment goes into a directory of your choosing and will > have its own python3 executable, and pip3 executable, and when you add > dependencies, they are also placed into the directory structure under > your chosen directory. > > When you do a `. /bin/activate` the included source will > places the virtual environment's bin/ folder at the beginning of your > PATH environment variable, making it the default python3 when you > type it without a full path. > > This allows you to run scripts that need different, or even > conflicting, sets of dependencies without bothering with the > underlying linux distribution's python installation's modules. > New to me. Interesting. > If you use `#!/usr/bin/python3`, it will always use exactly the system > version that is installed, and the system's installed modules. > > Your scripts will still default to the system installation if a > virtual environment is not activated. So you lose nothing by doing it > this way, but gain a little control from it. > Ok, understood. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Proper shebang for python3
Hi there, Pretty new to python I've got a question regarding the proper shebang for Python 3. I use #!/usr/bin/python3 which works fine. Today I saw #!/usr/bin/python3 -tt and was wondering what -tt means. Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even mention -t. python 2 man page mentions -t Issue a warning when a source file mixes tabs and spaces for indentation in a way that makes it depend on the worth of a tab expressed in spaces. Issue an error when the option is given twice. I guess that -t has the same meaning with python 3.7.3. My questions: 1. Is my guess correct? 2. Is it a bug that it is not mentioned? python3 --help doesn't mention it either. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Proper shebang for python3
On Sat, 20 Jul 2019 23:28:35 +0200 Brian Oney wrote: > Why not make a compromise? What would be a potential pitfall of the > following spitbang? > > #!python I think that per definition a path in a shebang has to be absolute. Actually, your suggestion won't work for people who use the fish shell (just for the given reason). -- https://mail.python.org/mailman/listinfo/python-list
Re: Proper shebang for python3
On Sun, 21 Jul 2019 10:21:55 +1000
Cameron Simpson wrote:
> On 21Jul2019 09:31, Chris Angelico wrote:
> >On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson
> >wrote: So you mean that a tool that depends on running on a
> >consistent environment, it should use a shebang of
> >"/usr/bin/python3.6" instead of "/usr/bin/env python3"?
>
> Jeez. No. That is the _opposite_ of what I'm saying.
>
> >Because, wow, that would be exactly what is
> >already happening on my system. Why use /usr/bin/env and then wrap
> >something around it to force the environment, when you could just set
> >a correct shebang so it properly defines its execution environment?
>
> Because the shebang is hardwired and inflexible.
>
> Because it means hand patching (even scripted) a bazillion scripts to
> that they know their physical install.
>
> Because it presumes detailed hardwired knowledge of the target system
> in a script which should work anywhere.
>
> Instead a tiny _common_ shell script resembling this:
>
> #!/bin/sh
> # Run command in the official environment.
> exec env - PATH=/path/to/3.6venv/bin:/usr/sbin:/bin exec ${1+"$@"}
>
> arranges things. The "env -" is aimed at "clean" daemon or install
> environments. You can do subtler or less intrusive things in other
> settings.
>
I took a look and found that Fedora 30 and Debian Jessie both use
hard-wired paths for python in the rpm resp. deb packages.
I'm being new to Python and I am not acquainted in any way with
virtualenv resp. venv so cannot currently judge its pro and cons.
So I will stick to:
#!/usr/bin/env python3
as shebang for my scripts.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: python socket dns query will get the correct result while the dig will not.
On Wed, 2 Oct 2019 04:27:14 + (UTC)
Hongyi Zhao wrote:
> Hi,
>
> See my following test:
>
> With ipython:
>
> In [1]: import
> socket
>
> In [2]: socket.gethostbyname
> ('www.vpngate.net')
> Out[2]: '130.158.75.44'
>
I get '130.158.75.39' which is ok.
>
> With dig:
>
> $ dig www.vpngate.net @114.114.114.114 +short
This gives here:
130.158.75.48
130.158.75.42
130.158.75.35
130.158.75.36
130.158.75.44
130.158.75.40
130.158.75.39
130.158.75.38
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Type checking
I had a look into type checking and tried a very simple example like the following: #!/usr/bin/env python3 def greeting(name: str) -> str: return 'Hello ' + name def main(): print(greeting(1)) if __name__ == "__main__": main() pytype complained as expected. BUT mypy says; Success: no issues found in 1 source file This is a Fedora 31 systems with: Python 3.7.4 mypy 0.730 Any idea why mypy doesn't detect anything? -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Type checking
On Sat, 19 Oct 2019 21:58:23 -0700 Paul Rubin wrote: > Manfred Lotz writes: > > def main(): > > print(greeting(1)) > > > > Any idea why mypy doesn't detect anything? > > > "main" doesn't have a type signature so it doesn't get checked. The > mypy docs explain this. Try: > > def main() -> None: > print(greeting(1)) Aah, such easy. As pytype did it I didn't assume something like this. Thanks a lot. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: "Don't install on the system Python"
On Sat, 30 Nov 2019 20:42:21 -0800 (PST) John Ladasky wrote: > Long-time Ubuntu user here. > > For years, I've read warnings about not installing one's personal > stack of Python modules on top of the system Python. It is possible > to corrupt the OS, or so I've gathered. > This is nonsense as you presumably have no permission to change anything Python related in /usr. The only possiblity I can imagine is that you somehow screw up your personal Python related setting in your home directory tree. But I have never (in the short period of time I've been using Python) encountered anything like this. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On Sun, 26 Apr 2020 15:26:58 +1200 DL Neil wrote: > On 25/04/20 7:53 PM, Manfred Lotz wrote: > > On Sat, 25 Apr 2020 18:41:37 +1200 > > DL Neil wrote: > > > >> On 25/04/20 5:16 PM, Manfred Lotz wrote: > >>> On Fri, 24 Apr 2020 19:12:39 -0300 > >>> Cholo Lennon wrote: > >>> > >>>> On 24/4/20 15:40, Manfred Lotz wrote: > >>>>> I have a command like application which checks a directory tree > >>>>> for certain things. If there are errors then messages will be > >>>>> written to stdout. > > > What I do here specifically is to check directory trees' file > > objects for user and group ownerships as well as permissions > > according to a given policy. There is a general policy and there > > could be exceptions which specify a specific policy for a certain > > file. > > If I have understood correctly, the objective is to check a dir-tree > to ensure that specific directory/file-permissions are in-effect/have > not been changed. The specifications come from a .JSON file and may > be over-ridden by command-line arguments. Correct? > Yes. > There must be a whole 'genre' of programs which inspect a > directory-tree and doing 'something' to the files-contained. I had a > few, and needing another, decided to write a generic 'scanner' which > would then call a tailored-function to perform the particular > 'something' - come to think of it, am not sure if that was ever quite > finished. Sigh! > > > >>>>> One idea was for the error situations to write messages to > >>>>> files and then later when running the tests to compare the > >>>>> error messages output to the previously saved output. > >>>>> > >>>>> Is there anything better? > > The next specification appears to be that you want a list of files > and their stats, perhaps reporting only any exceptions which weren't > there 'last time'. > I just want to report violations against the policy(ies) given in the JSON file. > In which case, an exception could be raised, or it might be simpler > to call a reporting-function when a file should be 'reported'. > > I'm still a little confused about the difference between > printing/logging/reporting some sort of exception, and the need to > compare with 'history'. > There is no compare with history. I just want to see current violations (I think I phrased things badly in my previous post so that it looked like I want to see history.) > > The problem with the "previously saved output" is the process of > linking 'today's data' with that from the last run. I would use a > database (but then, that's my background/bias) to store each file's > stat. > As said above I phrased things badly here in my previous post. > Alternately, if you only need to store exceptions, and that number is > likely to be small, perhaps output only the exceptions from 'this > run' to a .JSON/.yaml file - which would become input (and a dict for > easy look-ups) next time? > > > >>>> Maybe I am wrong because I don't understand your scenario: If > >>>> your application is like a command, it has to return an error > >>>> code to the system, a distinct number for each error condition. > >>>> The error code is easier to test than the stdout/stderr. > > Either way, you might decrease the amount of data to be stored by > reducing the file's stat to a code. > > > >>>>> How to test this in the best way? > > The best way to prepare for unit-testing is to have 'units' of code > (apologies!). > > An effective guide is to break the code into functions and methods, > so that each performs exactly one piece/unit of work - and only the > one. A better guide is that if you cannot name the procedure using > one description and want to add an "and" an "or" or some other > conjunction, perhaps there should be more than one procedure! > > For example: > > >for cat in cats: > > ... > > for d in scantree(cat.dir): > > # if `keep_fs` was specified then we must > > # make sure the file is on the same device > > if cat.keep_fs and devid != get_devid(d.path): > > continue > > > > cat.check(d) > Above is part of the main() function. But I could make some part of the main() function into its own function. Then the call to check a directory could be a function argument. For testing I could reuse that function by providing a diffe
How to test?
I have a command like application which checks a directory tree for certain things. If there are errors then messages will be written to stdout. How to test this in the best way? One idea was for the error situations to write messages to files and then later when running the tests to compare the error messages output to the previously saved output. Is there anything better? -- Thanks, Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On Mon, 27 Apr 2020 18:21:39 +1200 DL Neil wrote: ... > > Given your replies, 'now' might be a good time to take a look at > Pytest, and see how you could use it to help build better code - by > building tested units/functions which are assembled into ever-larger > tested-units... (there is a range of choice/other testing aids if > Pytest doesn't take your fancy) I have to admit I chose unittest. Simply because it is in the standard lbrary. As so many people seem to prefer pytest I should take a look at it. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On 24 Apr 2020 20:17:04 GMT r...@zedat.fu-berlin.de (Stefan Ram) wrote: > Manfred Lotz writes: > >I have a command like application which checks a directory tree for > >certain things. If there are errors then messages will be written to > >stdout. > > Error messages should be written to sys.stderr. > Yes. But for testing it doesn't matter if it is written to stdout or stderr. > >How to test this in the best way? > > The functions that check the tree should not write to > the console. Instead, when an error occurs they > > - raise an exception, This would be bad. In my case I want to report all the errors to somebody else. An exception means I don't see more errors which may have occured. > - yield an error information, Hm, you mean the check function would be something like an iterator. Sounds interesting. > - return an error information, or I think this is harder to implement. > - call a callback with the error information. > > Their callers than can be either high-level console code, > that writes those information to a console, or test code. > This is also very interesting. So, I think yield or callback are the ideas I have to investigate. Thanks a lot. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On Sat, 25 Apr 2020 08:25:00 +1200 DL Neil wrote: > On 25/04/20 6:40 AM, Manfred Lotz wrote: > > I have a command like application which checks a directory tree for > > certain things. If there are errors then messages will be written to > > stdout. > > > > How to test this in the best way? > > > > One idea was for the error situations to write messages to files and > > then later when running the tests to compare the error messages > > output to the previously saved output. > > > > Is there anything better? > > Yes, as well as reproducing the output on-screen, there are now three > ways to deal with stdout. The newest is "tee", which like the Linux > command of the same name, gives the best of both worlds - display and > 'capture'! > > Capturing of the stdout/stderr output > https://docs.pytest.org/en/latest/capture.html > > This is interesing. > May I point-out that the above may not be the best approach. Rather > than using screen-prints to report errors, another method is to > utilise "logging" to collect such data - so that there is always a > formal record (regardless of user behavior). During 'production' the > information could be collected at some central 'location' for > inspection by competent staff. During 'development', it is possible, > by changing one line, to re-direct the log to wherever you would like > - including the above! > Logging wouldn't help here as the person running the program is competent, and it likes to see what the errors are. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On Sat, 25 Apr 2020 18:41:37 +1200
DL Neil wrote:
> On 25/04/20 5:16 PM, Manfred Lotz wrote:
> > On Fri, 24 Apr 2020 19:12:39 -0300
> > Cholo Lennon wrote:
> >
> >> On 24/4/20 15:40, Manfred Lotz wrote:
> >>> I have a command like application which checks a directory tree
> >>> for certain things. If there are errors then messages will be
> >>> written to stdout.
> >>>
> >>> How to test this in the best way?
> >>>
> >>> One idea was for the error situations to write messages to files
> >>> and then later when running the tests to compare the error
> >>> messages output to the previously saved output.
> >>>
> >>> Is there anything better?
> >>>
> >>>
> >>
> >> Maybe I am wrong because I don't understand your scenario: If your
> >> application is like a command, it has to return an error code to
> >> the system, a distinct number for each error condition. The error
> >> code is easier to test than the stdout/stderr.
> >>
> >
> > Yes, a different error code for each condition is good to test.
> > However, I need to test as well the correct file name and other
> > values belonging to a certain error condition.
>
> It is frustrating to be shown only part of the information, and later
> be told our efforts aren't good-enough.
I found the tips I got here quite good. Actually, each time when I had
a questions here I got competent answers.
> How about respecting our
> (donated) time, and posting some sample code that shows exactly
> what/where the problem lies?
>
Sorry, perhaps I was too lazy to explain. Hope the following
explanation isn't too bad.
What I do here specifically is to check directory trees' file objects
for user and group ownerships as well as permissions according to a
given policy. There is a general policy and there could be exceptions
which specify a specific policy for a certain file.
The policy file is a JSON file and could have different categories.
Each category defines a policy for a certain directory tree. Comand
line args could overwrite directory names, as well as user and group.
Or if for example a directory is not specified in the JSON file I am
required to specify it via command line. Otherwise no check can take
place.
Here an example of a simple json file:
default":
{
"match_perms": "644",
"match_permsd": "755",
"match_permsx": "755",
"owner": "manfred",
"group": "manfred"
}
}
another simple example:
{
"default":
{
"have_all_permsx" : "555",
"have_all_perms" : "444",
"have_all_permsd" : "555",
"owner": "manfred",
"group": "manfred"
}
}
class Category contains a policy.
Here a simplified version of Category
class Category:
def __init__(self, cat, jsoncat, dirdict, ownerdict, groupdict):
self.cat = cat
self.jsoncat = jsoncat
self.keep_fs = jsoncat.get('keep_fs')
self.dir = jsoncat.get('dir')
if dirdict.get(cat):
self.dir = os.path.realpath(dirdict[cat])
self.policy = Policy(jsoncat, ownerdict, groupdict)
def check(self, fpath):
self.policy.check(fpath)
It is important to note that ownerdict and groupdict are overwrites
from the command line, and could be empty if nothing was
specified.
In my main routine I do this
...
for cat in cats:
...
for d in scantree(cat.dir):
# if `keep_fs` was specified then we must
# make sure the file is on the same device
if cat.keep_fs and devid != get_devid(d.path):
continue
cat.check(d)
...
class Policy contains required user and group owners and a
list of permission objects for files, for executable files and for
directories.
Those permissions are described like this
class BasePerms:
def __init__(self, perms: str):
"""Create an object holding a permission which is octal"""
self.perms = int(perms, 8)
class MatchPerms(BasePerms):
def check(self, fpath: str, mode: int):
mode = mode & 0o
if mode != self.perms:
print("%s perms differ, should have %s has %s" % (fpath,
oct(self.perms), oct(mode)))
class HaveAllPerms(BasePerms):
def check(self, fpath: str, mode: int):
mode = mode & 0o
if (mode & self.perms) != self.perms:
print("%s: perms differ, should have all of %s has %s" % (fpath,
oct(self.perms), oct(mode)))
After getting some ideas here I want to change my code to return an
error object in check() instead of doing a print(). Or return None
if all is ok.
Then the error object could have a print-like method and a compare
method so that later on when doing a test I could compare two error
objects. Perhaps I can combine it with callback. I have to play with
these things.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On 24 Apr 2020 22:18:45 GMT r...@zedat.fu-berlin.de (Stefan Ram) wrote: > DL Neil writes: > >Python's logging library enables messages to be formatted > >accordingly, and directed differently, depending upon 'level of > >severity'. So, as well as the flexibility mentioned before, there is > >an option to direct logging output to stdout/stderr as a > >matter-of-course! > > Here's some example code I wrote: > > import logging > > def setup_logging( logging ): > """Setup a logger using the standard logging module, > which needs to be passed as the argument""" > logger = logging.getLogger() > handler = logging.StreamHandler() > formatter = logging.Formatter( '%(asctime)s %(name)-12s > %(levelname)-8s %(message)s' ) handler.setFormatter( formatter ) > logger.addHandler( handler ) > logger.setLevel( logging.DEBUG ) > return logger > > logger = setup_logging( logging ) > logger.debug( "Hi!" ) > > It outputs (apparently to sys.stderr): > > 2020-04-24 23:13:59,467 root DEBUGHi! > > Yes, I know about this. But in this particular case I don't gain much by using a logger. print() is good enough. I will investigate yielding and callback to see which is best in my particular case. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On Fri, 24 Apr 2020 19:12:39 -0300 Cholo Lennon wrote: > On 24/4/20 15:40, Manfred Lotz wrote: > > I have a command like application which checks a directory tree for > > certain things. If there are errors then messages will be written to > > stdout. > > > > How to test this in the best way? > > > > One idea was for the error situations to write messages to files and > > then later when running the tests to compare the error messages > > output to the previously saved output. > > > > Is there anything better? > > > > > > Maybe I am wrong because I don't understand your scenario: If your > application is like a command, it has to return an error code to the > system, a distinct number for each error condition. The error code is > easier to test than the stdout/stderr. > Yes, a different error code for each condition is good to test. However, I need to test as well the correct file name and other values belonging to a certain error condition. -- https://mail.python.org/mailman/listinfo/python-list
ZipFile and timestamps
I unzip a zip file like follows which works fine. with ZipFile(zip_file, 'r') as zip: zip.extractall(tmp_dir) The only caveat is that the unpacked files and dirs do have a current timestamp i.e. the timestamps of the files are not preserved. Is there a way to tell ZipFile to preserve timestamps when unpacking a zip archive? -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: ZipFile and timestamps
On Wed, 18 Mar 2020 21:41:10 + MRAB wrote: > On 2020-03-18 20:38, Manfred Lotz wrote: > > I unzip a zip file like follows which works fine. > > > > with ZipFile(zip_file, 'r') as zip: > > zip.extractall(tmp_dir) > > > > The only caveat is that the unpacked files and dirs do have a > > current timestamp i.e. the timestamps of the files are not > > preserved. > > > > Is there a way to tell ZipFile to preserve timestamps when > > unpacking a zip archive? > > > You might have to iterate over the contents yourself and set the > modification times. Oops, if this is required then I better change my code to invoke unzip. Thanks, Manfred -- https://mail.python.org/mailman/listinfo/python-list
Strings: double versus single quotes
Hi there, I am asking myself if I should preferably use single or double quotes for strings? If I need a single quote in a string I would use double quotes for the whole string and vice versa. For f-strings I mostly see double quotes but single quotes would work as well I think. Is there a recommendation? -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Strings: double versus single quotes
On Tue, 19 May 2020 13:36:54 -0500 Tim Chase wrote: > On 2020-05-19 20:10, Manfred Lotz wrote: > > Hi there, > > I am asking myself if I should preferably use single or double > > quotes for strings? > > I'd say your consistency matters more than which one you choose. > > According to a recent observation by Raymond H. > > """ > Over time, the #python world has shown increasing preference > for double quotes: "hello" versus 'hello'. > > Perhaps, this is due to the persistent influence of JSON, > PyCharm, Black, and plain English. > > In contrast, the interpreter itself prefers single quotes: > > >>> "hello" > 'hello' > """ > > https://twitter.com/raymondh/status/1259209765072154624 > > I think the worst choice is to be haphazard in your usage with a > hodgepodge of single/double quotes. > > I personally use habits from my C days: double-quotes for everything > except single characters for which I use a single-quote: > > if 'e' in "hello": > > as in indicator that I'm using it as a single character rather than > as a string. > > I don't have a firm rule for myself if a string contains > double-quotes. It doesn't happen often for me in a case where I > couldn't use a triple-quoted string or that I am refering to it as a > single character. > > -tkc > > I am influenced by Perl which I used before. In Perl I used double quoted strings when I wanted interpolation. Otherwise single quoted strings. In Rust (or C) it is double quotes for strings and single quotes for characters. You may be right. Consistency is a good thing. So, I have to decide for what I use and be consistent thereafter. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Strings: double versus single quotes
On Sat, 23 May 2020 09:36:13 +1200 DL Neil wrote: > >>> I am asking myself if I should preferably use single or double > >>> quotes for strings? > ... > > > I agree to the following: > > > > 1. Consistency is important. > > 2. Single quotes are less noisy. > > 3. Triple double quotes are to be preferred over triple single > > quotes. > ... > > > Of course, this is my subjective result, and others (may) have > > different opinions. > > 4. unless a specific objective of a re-factoring exercise, maintain > the conventions used within the code. > > 5, adhere to the conventions/standards/requirements of the dev.team > or organisation. > Yes, agree. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Strings: double versus single quotes
On Sat, 23 May 2020 14:46:04 -0400
Dennis Lee Bieber wrote:
> On Sat, 23 May 2020 11:03:09 -0500, Tim Chase
> declaimed the following:
>
>
> >
> >But when a string contains both, it biases towards single quotes:
> >
> > >>> "You said \"No it doesn't\""
> > 'You said "No it doesn\'t"'
>
> This is where using triple quotes (or triple apostrophes)
> around the entire thing simplifies it all... (except for a need to
> separate the four ending quotes)
Exactly, I also would opt for triple quotes in this case.
>
> >>> """You said "No it doesn't" """
> 'You said "No it doesn\'t" '
> >>> '''You said "No it doesn't"'''
> 'You said "No it doesn\'t"'
> >>>
>
> NO \ escapes needed on the input strings.
>
> >>> print("""You said "No it doesn't" """)
> You said "No it doesn't"
> >>> print('''You said "No it doesn't"''')
> You said "No it doesn't"
> >>>
>
>
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
try..except or type() or isinstance()?
I have an object which I could initialize providind an int or a str. I am not sure which of the following is best to use - try/except - if type(int)... - if isinstance(v, int) Here a minimal example def get_id(fromname): # do something with `fromname` return 0 def get_name(fromid): # do something with `fromid` return "something" """ For O1, O2, O3: self.myid is int self.name is str """ class O1: def __init__(self, val): try: self.myid = int(val) self.name = get_name(self.myid) except: self.myid = get_id(val) self.name = val class O2: def __init__(self, val): if type(val) == int: self.myid = val self.name = get_name(self.myid) else: self.myid = get_id(val) self.name = val class O3: def __init__(self, val): if isinstance(val, int): self.myid = val self.name = get_name(self.myid) else: self.myid = get_id(val) self.name = val -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: try..except or type() or isinstance()?
On Sat, 15 Aug 2020 08:33:58 +0200 Peter Otten <__pete...@web.de> wrote: > Chris Angelico wrote: > > > On Sat, Aug 15, 2020 at 3:36 PM Manfred Lotz > > wrote: > >> > >> I have an object which I could initialize providind an int or a > >> str. > >> > >> I am not sure which of the following is best to use > >> - try/except > >> - if type(int)... > >> - if isinstance(v, int) > >> > >> Here a minimal example > >> > >> def get_id(fromname): > >> # do something with `fromname` > >> return 0 > >> > >> def get_name(fromid): > >> # do something with `fromid` > >> return "something" > >> > >> """ For O1, O2, O3: self.myid is int > >> self.name is str > >> """ > >> class O1: > >> def __init__(self, val): > >> try: > >> self.myid = int(val) > >> self.name = get_name(self.myid) > >> except: > >> self.myid = get_id(val) > >> self.name = val > > > > Don't use a bare "except" - use "except ValueError" instead. But > > otherwise, this is a perfectly reasonable way to say "anything that > > can be interpreted as an integer will be". > > > >> class O2: > >> def __init__(self, val): > >> if type(val) == int: > >> self.myid = val > >> self.name = get_name(self.myid) > >> else: > >> self.myid = get_id(val) > >> self.name = val > > > > Nope, don't do this. It's strictly worse than O3. > > > >> class O3: > >> def __init__(self, val): > >> if isinstance(val, int): > >> self.myid = val > >> self.name = get_name(self.myid) > >> else: > >> self.myid = get_id(val) > >> self.name = val > > > > This is a perfectly reasonable way to say "integers will be treated > > as IDs". Note that O1 and O3 are very different semantically; O1 > > will treat the string "7" as an ID, but O3 will treat it as a name. > > > > Here's an even better way: > > > > class O4: > > def __init__(self, id): > > self.myid = id > > self.name = get_name(id) > > @classmethod > > def from_name(cls, name): > > return cls(get_id(name)) > > > > This makes the ID the main way you'd do things, and a name lookup as > > an alternate constructor. Very good pattern, reliable, easy to use. > > > > Yet another way: keyword arguments: > > class O5: > def __init__(self, *, id=None, name=None): > if name is None: > assert id is not None > name = get_name(id) > else: > assert id is None > id = get_id(name) > self.id = id > self.name = name > Thanks. Also a nice method. As said in my other reply it doesn't fit to my use case. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: try..except or type() or isinstance()?
On Sat, 15 Aug 2020 11:47:03 +0200 Sibylle Koczian wrote: > Am 15.08.2020 um 10:14 schrieb Manfred Lotz: > > Hi Chris, > > Thanks a lot for you advice. > > > > On Sat, 15 Aug 2020 16:15:29 +1000 > > Chris Angelico wrote: > > > >> On Sat, Aug 15, 2020 at 3:36 PM Manfred Lotz > >> wrote: > >>> > >>> I have an object which I could initialize providind an int or a > >>> str. > ... > > > > For my use case I don't like this solution as I get the value which > > could be an int or str from a file. Means I would have to check the > > type beforehand in order to know if I have to do O4(val) or > > O4.from_name(val). > > > > Otherwise, it is admittedly a very good pattern. > > > > > Possibly silly question: No silly questions. There are at most stupid answers. :-) > if the value comes from a file, isn't it a > string in any case? A string that may be convertible to int or not? Or > what sort of file do I overlook? > In this case it is a TOML file. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: try..except or type() or isinstance()?
Hi Chris, Thanks a lot for you advice. On Sat, 15 Aug 2020 16:15:29 +1000 Chris Angelico wrote: > On Sat, Aug 15, 2020 at 3:36 PM Manfred Lotz > wrote: > > > > I have an object which I could initialize providind an int or a str. > > > > I am not sure which of the following is best to use > > - try/except > > - if type(int)... > > - if isinstance(v, int) > > > > Here a minimal example > > > > def get_id(fromname): > > # do something with `fromname` > > return 0 > > > > def get_name(fromid): > > # do something with `fromid` > > return "something" > > > > """ For O1, O2, O3: self.myid is int > > self.name is str > > """ > > class O1: > > def __init__(self, val): > > try: > > self.myid = int(val) > > self.name = get_name(self.myid) > > except: > > self.myid = get_id(val) > > self.name = val > > Don't use a bare "except" - use "except ValueError" instead. Oops, yes. You are right, of course. > But > otherwise, this is a perfectly reasonable way to say "anything that > can be interpreted as an integer will be". > > > class O2: > > def __init__(self, val): > > if type(val) == int: > > self.myid = val > > self.name = get_name(self.myid) > > else: > > self.myid = get_id(val) > > self.name = val > > Nope, don't do this. It's strictly worse than O3. > Aaa ok. I think in this particular case it would be ok but in general it might be bad if val would be a class object. So, I take your advice to avoid this in general as then I avoid errors in more complicated situations when using type(..). > > class O3: > > def __init__(self, val): > > if isinstance(val, int): > > self.myid = val > > self.name = get_name(self.myid) > > else: > > self.myid = get_id(val) > > self.name = val > > This is a perfectly reasonable way to say "integers will be treated as > IDs". Note that O1 and O3 are very different semantically; O1 will > treat the string "7" as an ID, but O3 will treat it as a name. > > Here's an even better way: > > class O4: > def __init__(self, id): > self.myid = id > self.name = get_name(id) > @classmethod > def from_name(cls, name): > return cls(get_id(name)) > > This makes the ID the main way you'd do things, and a name lookup as > an alternate constructor. Very good pattern, reliable, easy to use. > For my use case I don't like this solution as I get the value which could be an int or str from a file. Means I would have to check the type beforehand in order to know if I have to do O4(val) or O4.from_name(val). Otherwise, it is admittedly a very good pattern. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Use of a variable in parent loop
On Mon, 28 Sep 2020 05:20:20 +0800 Stephane Tougard wrote: > On 2020-09-27, Manfred Lotz wrote: > > - http://localhost:2015/tutorial/controlflow.html#pass-statements > ... > > (In comparison to guys like ChrisA and StefanR and others here I am > > also a Python beginner) > > To give me a pointer on your localhost, I could guess. Don't understand this sentence. -- https://mail.python.org/mailman/listinfo/python-list
Re: Use of a variable in parent loop
On Mon, 28 Sep 2020 13:39:08 +0800 Stephane Tougard wrote: > On 2020-09-28, Manfred Lotz wrote: > > On Mon, 28 Sep 2020 05:20:20 +0800 > > Stephane Tougard wrote: > > > >> On 2020-09-27, Manfred Lotz wrote: > >> > - > >> > http://localhost:2015/tutorial/controlflow.html#pass-statements > >> > > >> ... > >> > (In comparison to guys like ChrisA and StefanR and others here I > >> > am also a Python beginner) > >> > >> To give me a pointer on your localhost, I could guess. > > > > Don't understand this sentence. > > The URL you gave is pointing to your localhost, your own computer if > you prefer. It is amazing that I just didn't see it. I am a bad at spotting things. Sigh. Anyway: https://docs.python.org/3/tutorial/controlflow.html#pass-statements -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Use of a variable in parent loop
On Sun, 27 Sep 2020 15:18:44 +0800 Stephane Tougard wrote: > On 2020-09-27, 2qdxy4rzwzuui...@potatochowder.com > <2qdxy4rzwzuui...@potatochowder.com> wrote: > > As ChrisA noted, Python almost always Just Works without > > declarations. If you find yourself with a lot of global and/or > > nonlocal statements, perhaps you're [still] thinking in another > > language. > > > I don't really agree with that, trying to use an undeclared > object/variable/whatever : > > Python 3.7.7 (default, Aug 22 2020, 17:07:43) > [GCC 7.4.0] on netbsd9 > Type "help", "copyright", "credits" or "license" for more information. > >>> print(name) > Traceback (most recent call last): > File "", line 1, in > NameError: name 'name' is not defined > >>> > > You can say it's not the "declaration" the issue, it's the > "definition", that's just a matter of vocabulary and it does not > answer the question. > > In many non declarative language, if I do print($var), it just prints > and undefined value with returning an error. > > It is very good that you try out things. Nevertheless, it is also good to read. In other words the combination of reading and trying out things is the best way to make progress. Here some pointers pass - http://localhost:2015/tutorial/controlflow.html#pass-statements about variables - https://realpython.com/python-variables/ Regarding your question above: As long as `name` is not attached to an object it doesn't exist and the error message above is correct. You have to assign a value (or object as everything is an object in Python) to `name`, Then you can use it. (In comparison to guys like ChrisA and StefanR and others here I am also a Python beginner) -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Writing output of function to a csv file
On Mon, 25 May 2020 19:25:08 +0100
MRAB wrote:
> On 2020-05-25 19:00, Ciarán Hudson wrote:
> > Hi,
> >
> > In the code below, which is an exercise I'm doing in class
> > inheritance, almost everything is working except outputting my
> > results, which as the function stock_count, to a csv. stock_count
> > is working in the code. And I'm able to open and close the csv
> > called cars, but the stock_count output is not being written to the
> > file. Any suggestions?
> [snip]
> >
> > def stock_count(self):
> > print('petrol cars in stock ' + str(len(self.petrol_cars)))
> > print('electric cars in stock ' +
> > str(len(self.electric_cars))) print('diesel cars in stock ' +
> > str(len(self.diesel_cars))) print('hybrid cars in stock ' +
> > str(len(self.hybrid_cars)))
> >
> > def process_rental(self):
> > answer = input('would you like to rent a car? y/n')
> > if answer == 'y':
> > self.stock_count()
> > answer = input('what type would you like? p/e/d/h')
> > amount = int(input('how many would you like?'))
> > if answer == 'p':
> > self.rent(self.petrol_cars, amount)
> > if answer == 'd':
> > self.rent(self.diesel_cars, amount)
> > if answer == 'h':
> > self.rent(self.hybrid_cars, amount)
> > else:
> > self.rent(self.electric_cars, amount)
> > self.stock_count()
> >
> > file = open("cars.csv","w")
> > file.write(str(self.stock_count()))
> > file.close()
> >
> In 'stock_count' you're telling it to print to the screen. Nowhere in
> that function are you telling it to write to a file.
I think something gets written to 'cars.csv', namely the string 'None'.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: Strings: double versus single quotes
On Tue, 19 May 2020 13:36:54 -0500 Tim Chase wrote: > On 2020-05-19 20:10, Manfred Lotz wrote: > > Hi there, > > I am asking myself if I should preferably use single or double > > quotes for strings? > > I'd say your consistency matters more than which one you choose. > First, thanks to all for your opinions and judgements. I agree to the following: 1. Consistency is important. 2. Single quotes are less noisy. 3. Triple double quotes are to be preferred over triple single quotes. I also believe that transferring habits from C, Rust or whatever to Python doesn't make much sense as Python does not distinguish between a character and string from a type perspective. Of course, this is my subjective result, and others (may) have different opinions. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to test?
On Wed, 17 Jun 2020 17:34:55 +0100
Tony Flury wrote:
> On 24/04/2020 19:40, Manfred Lotz wrote:
> > I have a command like application which checks a directory tree for
> > certain things. If there are errors then messages will be written to
> > stdout.
> >
> > How to test this in the best way?
> >
> > One idea was for the error situations to write messages to files and
> > then later when running the tests to compare the error messages
> > output to the previously saved output.
> >
> > Is there anything better?
> >
>
> In a recent application that I wrote (where output to the console was
> important), I tested it using the 'unittest' framework, and by
> patching sys.stderr to be a StringIO - that way my test case could
> inspect what was being output.
>
> with patch('sys.stderr', StringIO()) as
> stderr:application.do_stuff()self.assertTrue(stderr.getvalue(),
> 'Woops - that didn\'t work')
>
> I am not sure of the structure of your application, and whether you
> have a callable API that you can invoke.
>
Thanks a lot. That's really great.
In the meantime I switched to pytest because things are so much more
convenient to setup. So, also thanks to the others who gave the links to
how this can be done with pytest.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: Syntax question
On Sun, 16 Aug 2020 10:12:04 +0200 Klaus Jantzen wrote: > Hi, > > the other day I came across the book "Classic Computer Science > Problems in Python" by David Kopec. > > The function definitions in the examples like > > = > def fib2(n: int) -> int: > if n < 2: # base case > return n > return fib2(n - 2) + fib2(n - 1) # recursive case > > > if __name__ == "__main__": > print(fib2(5)) > print(fib2(10)) > > = > > use a syntax that I have never seen on this list or in other > publications. > What do you mean? The 'n:int' and '-> int'? If yes, this is type hinting. See here: https://docs.python.org/3/library/typing.html -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: try..except or type() or isinstance()?
On 15 Aug 2020 14:49:48 GMT r...@zedat.fu-berlin.de (Stefan Ram) wrote: > Manfred Lotz writes: > >Here a minimal example > > main.py > > source=""" > sehr gut > 1 > """[ 1: -1 ].split( "\n" ) > > class grades: > names =[ "sehr gut" ] > @staticmethod > def is_numeric( text ): > return text.isdigit() > @staticmethod > def get_number( text ): > return grades.names.index( text )+ 1 > @staticmethod > def get_name( text ): > return grades.names[ int( text )- 1 ] > > class O1: > def init_from_number( self, text ): > self.myid = int( text ) > self.name = grades.get_name( text ) > def init_from_name( self, text ): > self.myid = grades.get_number( text ) > self.name = text > def __init__( self, text ): > if grades.is_numeric( text ): > self.init_from_number( text ) > else: > self.init_from_name( text ) > def __str__( self ): > return "O1( " + str( self.myid )+ ", " + str( self.name ) + " > )" > for line in source: > print( O1( line )) > > > transcript: > > O1( 1, sehr gut ) > O1( 1, sehr gut ) > > Thanks for this interesting variation. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Syntax question
On Sun, 16 Aug 2020 06:09:17 -0500 Skip Montanaro wrote: > > Typing is not required by > > Python. However, you may find the extra error-checking helpful... > > I haven't used type hints much, if at all, but my understanding is > that the "extra error-checking" of which you speak is gotten through > other static checkers, correct? Yes. > I know the syntax was developed with > the MyPy folks (http://mypy-lang.org/), but I'm not sure what other > tools currently make use of it. Any pointers? > I heard about the following (not tried all, though) - mypy (Dropbox) - pyre-check (Facebook) - pytype (Google) - pyright (Microsoft) -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: try..except or type() or isinstance()?
On Sun, 16 Aug 2020 09:40:12 +0200
Manfred Lotz wrote:
> On Sat, 15 Aug 2020 12:20:48 -0400
> Dennis Lee Bieber wrote:
>
> > On Sat, 15 Aug 2020 15:31:56 +0200, Manfred Lotz
> > declaimed the following:
> >
> > >On Sat, 15 Aug 2020 11:47:03 +0200
> > >Sibylle Koczian wrote:
> > >
> >
> > >> if the value comes from a file, isn't it a
> > >> string in any case? A string that may be convertible to int or
> > >> not? Or what sort of file do I overlook?
> > >>
> > >
> > >In this case it is a TOML file.
> >
> > Based on https://en.wikipedia.org/wiki/TOML this means you
> > need a parser...
> > """
> > In TOML the syntax determines the data types ("syntax typing")
> > """
> >
> > String data is surrounded with " marks, numbers are not
> > (though the example doesn't show if integers are treated differently
> > from floating point), arrays/lists in [] with embedded comma ([] are
> > also overloaded for section headers, with subsections using
> > section.subsection naming), dates are some ugly creation, and looks
> > like true/false are reserved values.
> >
> > However, as pointed out -- all data read from the file will
> > be seen as a Python string data type. It is only after determining
> > the TOML data type -- by examining the string itself -- that one can
> > convert to internal format.
> >
> > Unfortunately, TOML is not compatible with INI -- for which
> > Python already has a read/write module. But there is
> > https://pypi.org/project/toml/
> > (uses the same example file as Wikipedia) -- latest version was from
> > May.
> >
> >
>
> I use tomlkit as the toml package doesn't support quoted keys
> sufficiently.
>
Just checked again. It seems I was wrong. Both toml and tomlkit do
support quoted keys ok.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: try..except or type() or isinstance()?
On Sat, 15 Aug 2020 12:20:48 -0400
Dennis Lee Bieber wrote:
> On Sat, 15 Aug 2020 15:31:56 +0200, Manfred Lotz
> declaimed the following:
>
> >On Sat, 15 Aug 2020 11:47:03 +0200
> >Sibylle Koczian wrote:
> >
>
> >> if the value comes from a file, isn't it a
> >> string in any case? A string that may be convertible to int or
> >> not? Or what sort of file do I overlook?
> >>
> >
> >In this case it is a TOML file.
>
> Based on https://en.wikipedia.org/wiki/TOML this means you
> need a parser...
> """
> In TOML the syntax determines the data types ("syntax typing")
> """
>
> String data is surrounded with " marks, numbers are not
> (though the example doesn't show if integers are treated differently
> from floating point), arrays/lists in [] with embedded comma ([] are
> also overloaded for section headers, with subsections using
> section.subsection naming), dates are some ugly creation, and looks
> like true/false are reserved values.
>
> However, as pointed out -- all data read from the file will
> be seen as a Python string data type. It is only after determining
> the TOML data type -- by examining the string itself -- that one can
> convert to internal format.
>
> Unfortunately, TOML is not compatible with INI -- for which
> Python already has a read/write module. But there is
> https://pypi.org/project/toml/
> (uses the same example file as Wikipedia) -- latest version was from
> May.
>
>
I use tomlkit as the toml package doesn't support quoted keys
sufficiently.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Changing strings in files
I have a situation where in a directory tree I want to change a certain string in all files where that string occurs. My idea was to do - os.scandir and for each file - check if a file is a text file - if it is not a text file skip that file - change the string as often as it occurs in that file What is the best way to check if a file is a text file? In a script I could use the `file` command which is not ideal as I have to grep the result. In Perl I could do -T file. How to do best in Python? -- Thanks, Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Changing strings in files
On Tue, 10 Nov 2020 18:52:26 +1100
Mike Dewhirst wrote:
> On 10/11/2020 5:24 pm, Manfred Lotz wrote:
> > I have a situation where in a directory tree I want to change a
> > certain string in all files where that string occurs.
> >
> > My idea was to do
> >
> > - os.scandir and for each file
> > - check if a file is a text file
> > - if it is not a text file skip that file
> > - change the string as often as it occurs in that file
> >
> >
> > What is the best way to check if a file is a text file? In a script
> > I could use the `file` command which is not ideal as I have to grep
> > the result. In Perl I could do -T file.
> >
> > How to do best in Python?
> Not necessarily best but I rolled this earlier. Much earlier. But I
> still use it. I don't need to worry about text files or binary
> because I specify .py files.
>
>
>
> # -*- coding: utf-8 -*-
> """
> Look for string 'find' and replace it with string 'repl' in all files
> in the current directory and all sub-dirs.
>
> If anything untoward happens, laboriously retrieve each original file
> from each individual backup made by suffixing ~ to the filename.
>
> If everything went well, make find == repl and run again to remove
> backups.
>
> """
> import os
>
> find = """# Copyright (c) 2019 Xyz Pty Ltd"""
>
> repl = """# Copyright (c) 2020 Xyz Pty Ltd"""
>
> ii = 0
> kk = 0
> for dirpath, dirnames, filenames in os.walk(".", topdown=True):
> if "migrations" in dirpath:
> continue
> for filename in filenames:
> if filename.endswith(".py"): # or filename.endswith(".txt"):
> fil = os.path.join(dirpath, filename)
> bak = "{0}~".format(fil)
> if find == repl:
> if os.path.isfile(bak):
> ii += 1
> os.remove(bak)
> else:
> with open(fil, "r") as src:
> lines = src.readlines()
> # make a backup file
> with open(bak, "w", encoding="utf-8") as dst:
> for line in lines:
> dst.write(line)
> with open(bak, "r") as src:
> lines = src.readlines()
> # re-write the original src
> with open(fil, "w", encoding="utf-8") as dst:
> kk += 1
> for line in lines:
> dst.write(line.replace(find, repl))
> print("\nbak deletions = %s, tried = %s\n" % (ii, kk))
>
>
Thanks, will take a look.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: Changing strings in files
On Tue, 10 Nov 2020 18:37:54 +1100 Cameron Simpson wrote: > On 10Nov2020 07:24, Manfred Lotz wrote: > >I have a situation where in a directory tree I want to change a > >certain string in all files where that string occurs. > > > >My idea was to do > > > >- os.scandir and for each file > > Use os.walk for trees. scandir does a single directory. > Perhaps better. I like to use os.scandir this way def scantree(path: str) -> Iterator[os.DirEntry[str]]: """Recursively yield DirEntry objects (no directories) for a given directory. """ for entry in os.scandir(path): if entry.is_dir(follow_symlinks=False): yield from scantree(entry.path) yield entry Worked fine so far. I think I coded it this way because I wanted the full path of the file the easy way. > > - check if a file is a text file > > This requires reading the entire file. You want to check that it > consists entirely of lines of text. In your expected text encoding - > these days UTF-8 is the common default, but getting this correct is > essential if you want to recognise text. So as a first cut, totally > untested: > > ... The reason I want to check if a file is a text file is that I don't want to try replacing patterns in binary files (executable binaries, archives, audio files aso). Of course, to make this nicely work some heuristic check would be the right thing (this is what file command does). I am aware that an heuristic check is not 100% but I think it is good enough. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Changing strings in files
On Tue, 10 Nov 2020 08:19:55 +0100
"Loris Bennett" wrote:
> Manfred Lotz writes:
>
> > I have a situation where in a directory tree I want to change a
> > certain string in all files where that string occurs.
> >
> > My idea was to do
> >
> > - os.scandir and for each file
> >- check if a file is a text file
> >- if it is not a text file skip that file
> >- change the string as often as it occurs in that file
> >
> >
> > What is the best way to check if a file is a text file? In a script
> > I could use the `file` command which is not ideal as I have to grep
> > the result. In Perl I could do -T file.
> >
> > How to do best in Python?
>
> If you are on Linux and more interested in the result than the
> programming exercise, I would suggest the following non-Python
> solution:
>
>find . -type -f -exec sed -i 's/foo/bar/g' {} \;
>
My existing script in Perl which I wanted to migrate to Python I used
`-T $file` and called sed
I like the -T which I assume does some heuristics to tell me if a file
is a text file.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: Changing strings in files
On Tue, 10 Nov 2020 22:08:54 +1100 Cameron Simpson wrote: > On 10Nov2020 10:07, Manfred Lotz wrote: > >On Tue, 10 Nov 2020 18:37:54 +1100 > >Cameron Simpson wrote: > >> Use os.walk for trees. scandir does a single directory. > > > >Perhaps better. I like to use os.scandir this way > > > >def scantree(path: str) -> Iterator[os.DirEntry[str]]: > >"""Recursively yield DirEntry objects (no directories) > > for a given directory. > >""" > >for entry in os.scandir(path): > >if entry.is_dir(follow_symlinks=False): > >yield from scantree(entry.path) > > > >yield entry > > > >Worked fine so far. I think I coded it this way because I wanted the > >full path of the file the easy way. > > Yes, that's fine and easy to read. Note that this is effectively a > recursive call though, with the associated costs: > > - a scandir (or listdir, whatever) has the directory open, and holds > it open while you scan the subdirectories; by contrast os.walk only > opens one directory at a time > > - likewise, if you're maintaining data during a scan, that is held > while you process the subdirectories; with an os.walk you tend to do > that and release the memory before the next iteration of the main > loop (obviously, depending exactly what you're doing) > > However, directory trees tend not to be particularly deep, and the > depth governs the excess state you're keeping around. > Very interesting information. Thanks a lot for this. I will take a closer look at os.walk. > >> > - check if a file is a text file > >> > >> This requires reading the entire file. You want to check that it > >> consists entirely of lines of text. In your expected text encoding > >> - these days UTF-8 is the common default, but getting this correct > >> is essential if you want to recognise text. So as a first cut, > >> totally untested: > >> > >> ... > > > >The reason I want to check if a file is a text file is that I don't > >want to try replacing patterns in binary files (executable binaries, > >archives, audio files aso). > > Exactly, which is why you should not trust, say, the "file" utility. > It scans only the opening part of the file. Great for rejecting > files, but not reliable for being _sure_ about the whole file being > text when it doesn't reject. > > >Of course, to make this nicely work some heuristic check would be the > >right thing (this is what file command does). I am aware that an > >heuristic check is not 100% but I think it is good enough. > > Shrug. That is a risk you must evaluate yourself. I'm quite paranoid > about data loss, myself. If you've got backups or are working on > copies the risks are mitigated. > > You could perhaps take a more targeted approach: do your target files > have distinctive file extensions (for example, all the .py files in a > source tree). > There are some distinctive file extensions. The reason I am satisfieg with heuristics is that the string to change is pretty long so that there is no real danger if I try to change in a binary file because that string it not to be found in binary files. The idea to skip binary files was simply to save time. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Changing strings in files
On Tue, 10 Nov 2020 10:57:05 +0100
"Loris Bennett" wrote:
> Manfred Lotz writes:
>
> > On Tue, 10 Nov 2020 08:19:55 +0100
> > "Loris Bennett" wrote:
> >
> >> Manfred Lotz writes:
> >>
> >> > I have a situation where in a directory tree I want to change a
> >> > certain string in all files where that string occurs.
> >> >
> >> > My idea was to do
> >> >
> >> > - os.scandir and for each file
> >> >- check if a file is a text file
> >> >- if it is not a text file skip that file
> >> >- change the string as often as it occurs in that file
> >> >
> >> >
> >> > What is the best way to check if a file is a text file? In a
> >> > script I could use the `file` command which is not ideal as I
> >> > have to grep the result. In Perl I could do -T file.
> >> >
> >> > How to do best in Python?
> >>
> >> If you are on Linux and more interested in the result than the
> >> programming exercise, I would suggest the following non-Python
> >> solution:
> >>
> >>find . -type -f -exec sed -i 's/foo/bar/g' {} \;
> >>
> >
> > My existing script in Perl which I wanted to migrate to Python I
> > used `-T $file` and called sed
> >
> > I like the -T which I assume does some heuristics to tell me if a
> > file is a text file.
>
> Sorry, I missed the bit about text files. By '-T' I assume you mean
> Perl's taint mode option. I am no security expert, but as I
> understand it, taint mode does more than just check whether something
> is a text file, although the "more" probably applies mainly to files
> which contain Perl code.
>
> Sorry also to bang on about non-Python solutions but you could do
>
> find . -type f -exec grep -Iq . {} \; -and -exec sed -i
> 's/foo/bar/g' {} \;
>
> i.e. let grep ignore binary files and quietly match all non-binary
> files.
>
Very nice. Thanks for this.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: Class Definitions
On 11 Nov 2020 19:21:57 GMT r...@zedat.fu-berlin.de (Stefan Ram) wrote: > In my Python course I gave the assignment to define a > counter class "Main" so that > > counter0 = Main() > counter1 = Main() > counter1.count(); counter1.count(); counter1.count() > counter1.count(); counter1.count() > print( counter0.value ) > print( counter1.value ) > > would print > > 0 > 5 > > . > > I expected this solution: > > class Main: > def __init__( self ): > self.value = 0 > def count( self ): > self.value += 1 > > but a student turned in the following solution: > > class Main: > value = 0 > def count(self): > self.value += 1 > > . I am still a Python beginner and didn't even believe that the student's solution would work. I had expected an error as the instance variable self.value was not initialized. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Class Definitions
On Sat, 14 Nov 2020 05:08:07 -0600 2qdxy4rzwzuui...@potatochowder.com wrote: > On 2020-11-14 at 10:09:32 +0100, > Manfred Lotz wrote: > > > On 11 Nov 2020 19:21:57 GMT > > r...@zedat.fu-berlin.de (Stefan Ram) wrote: > > > > > In my Python course I gave the assignment to define a > > > counter class "Main" so that > > > > > > counter0 = Main() > > > counter1 = Main() > > > counter1.count(); counter1.count(); counter1.count() > > > counter1.count(); counter1.count() > > > print( counter0.value ) > > > print( counter1.value ) > > > > > > would print > > > > > > 0 > > > 5 > > > > > > . > > > > > > I expected this solution: > > > > > > class Main: > > > def __init__( self ): > > > self.value = 0 > > > def count( self ): > > > self.value += 1 > > > > > > but a student turned in the following solution: > > > > > > class Main: > > > value = 0 > > > def count(self): > > > self.value += 1 > > > > > > . > > > > I am still a Python beginner and didn't even believe that the > > student's solution would work. I had expected an error as the > > instance variable self.value was not initialized. > > Remember: (1) x += 1 behaves like x = x + 1, and (2) bindings created > inside a class statement but outside any method create class > attributes. > > So after counter0 = Main(), Main (the class) has an attribute called > "value" whose value is 0, and counter0.value refers to that attribute. > > Then counter0.count() executes self.value += 1, which behaves like > self.value = self.value + 1. The right side of that assignment > evaluates to 1 (the value of the class attribute plus the constant 1), > and then the assignment statement initializes self.value to *that* > value. > > There's nothing special about initializing instance attributes in > __init__. An instance attribute can be created/initialized anywhere. > > Thanks to you and Stefan. I did not know that if an instance variable doesn't exist the same name is searched for as a class variable. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: learning python ...
On Mon, 24 May 2021 08:14:39 +1200 Ron.Lauzon@f192.n1.z21.fsxnet (Ron Lauzon) wrote: > -=> hw wrote to All <=- > > hw> Traceback (most recent call last): > hw>File "[...]/hworld.py", line 18, in > hw> print(isinstance(int, float)) > hw> TypeError: isinstance() arg 2 must be a type or tuple of types > > hw> I would understand to get an error message in line 5 but not in > hw> 18. Is this a bug or a feature? > > Python is case sensitive. Is that supposed to be "isInstance"? > > This is easy to check $ python3 Python 3.9.5 (default, May 14 2021, 00:00:00) [GCC 11.1.1 20210428 (Red Hat 11.1.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> isinstance >>> isInstance Traceback (most recent call last): File "", line 1, in NameError: name 'isInstance' is not defined -- https://mail.python.org/mailman/listinfo/python-list
Choosable dependency
Let us say I have a package which reads a TOML file. I want to give the user of my package the choice to decide if he wants to use the toml, tomlkit or rtoml package. So, in case the user chose to use rtoml then there should be an import only for rtoml, aso. How could I achieve this? -- Thanks, Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Choosable dependency
On Sat, 6 Mar 2021 12:00:33 +0100
Manfred Lotz wrote:
> Let us say I have a package which reads a TOML file.
>
> I want to give the user of my package the choice to decide if he wants
> to use the toml, tomlkit or rtoml package.
>
> So, in case the user chose to use rtoml then there should be an import
> only for rtoml, aso.
>
> How could I achieve this?
>
I got the following minimal example working but presumably my solution
isn't the best way.
xtoml.py:
class Xtoml:
def __init__(self, which='rtoml'):
self.which = which
def parse_toml(self, toml_string):
if self.which == 'rtoml':
import rtoml as toml
elif self.which == 'tomlkit':
import tomlkit as toml
else:
import toml
return toml.loads(toml_string)
test_xtoml.py:
from xtoml import Xtoml
toml_string = """
[default]
basedir = "/myproject"
"""
def main():
xtoml = Xtoml('toml')
parsed_toml = xtoml.parse_toml(toml_string)
print(parsed_toml)
xtoml = Xtoml()
parsed_toml = xtoml.parse_toml(toml_string)
print(parsed_toml)
if __name__ == "__main__":
main()
--
https://mail.python.org/mailman/listinfo/python-list
Re: Choosable dependency
On Sat, 6 Mar 2021 14:06:27 +0100
Peter Otten <__pete...@web.de> wrote:
> On 06/03/2021 12:43, Manfred Lotz wrote:
> > On Sat, 6 Mar 2021 12:00:33 +0100
> > Manfred Lotz wrote:
> >
> >> Let us say I have a package which reads a TOML file.
> >>
> >> I want to give the user of my package the choice to decide if he
> >> wants to use the toml, tomlkit or rtoml package.
> >>
> >> So, in case the user chose to use rtoml then there should be an
> >> import only for rtoml, aso.
> >>
> >> How could I achieve this?
> >>
> >
> > I got the following minimal example working but presumably my
> > solution isn't the best way.
> >
> > xtoml.py:
> >
> > class Xtoml:
> > def __init__(self, which='rtoml'):
> > self.which = which
> >
> > def parse_toml(self, toml_string):
> > if self.which == 'rtoml':
> > import rtoml as toml
> > elif self.which == 'tomlkit':
> > import tomlkit as toml
> > else:
> > import toml
> >
> > return toml.loads(toml_string)
> >
> >
> > test_xtoml.py:
> >
> > from xtoml import Xtoml
> >
> > toml_string = """
> > [default]
> >
> > basedir = "/myproject"
> >
> > """
> >
> > def main():
> > xtoml = Xtoml('toml')
> > parsed_toml = xtoml.parse_toml(toml_string)
> > print(parsed_toml)
> >
> > xtoml = Xtoml()
> > parsed_toml = xtoml.parse_toml(toml_string)
> > print(parsed_toml)
> >
> > if __name__ == "__main__":
> > main()
>
> As long as you use a common subset of the functions provided by the
> packages you can just do
>
Yes, this is a prereq. Otherwise, it wouldn't make sense to do this.
> def main():
> global toml # if you want to use the toml package elsewhere
> # in your module
>
> chosen_toml = ... # get package name
> toml = importlib.import_module(chosen_toml)
>
> data = toml.loads(toml_string)
>
> However, the user usually doesn't care, so I would recommend that you
> be bold and pick the package you prefer ;)
>
Exactly. This is why there is a default.
Using your suggestion my minimal example now looks like follows
xtoml.py:
import importlib
class Xtoml:
def __init__(self, which='rtoml'):
self.toml = importlib.import_module(which)
def parse_toml(self, toml_string):
return self.toml.loads(toml_string)
No change in test_xtoml.py.
Thanks a lot for your help.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: Choosable dependency
On Sat, 6 Mar 2021 15:40:41 - (UTC) Grant Edwards wrote: > On 2021-03-06, Manfred Lotz wrote: > > Let us say I have a package which reads a TOML file. > > > > I want to give the user of my package the choice to decide if he > > wants to use the toml, tomlkit or rtoml package. > > Why would the user choose one over the other? Does the user get > different functions/features depending on which one they choose? > I am a user of my package and I like to use rtoml which however is not available on another OS (due to Rust no being available there) where I also want to use my package. Here I can use toml which is always possible to install. Therefore, toml is the default so that the ordinary user doesn't need to care. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: Choosable dependency
On Sun, 7 Mar 2021 20:38:30 -0800 Dan Stromberg wrote: > I sometimes do things like: > > try: > import rtoml as toml > except ImportError: > import toml > Also a possibility. Thanks. > ...but I don't like it very much, because it tends to confuse static > analyzers like pyflakes and pylint. Maybe mypy will deal with it > better? Not sure yet. > Good to know that there could be problems here. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
On Thu, 9 Dec 2021 18:18:26 +0100
"Dieter Maurer" wrote:
> Manfred Lotz wrote at 2021-12-8 12:31 +0100:
> >The are many possibilities to package a Python app, and I have to
> >admit I am pretty confused.
> >
> >Here is what I have:
> >
> >A Python command line app which requires some packages which are not
> >in the standard library.
>
> Are they on PyPI or can they be downloaded to PyPI?
> In this case, you could install it via `pip` and
> a "requirements" file (describing what is necessary).
>
Acutally, those packages are on pypi.
I created a minimal example where I have two files
hello.py
#!/usr/bin/python3
from message import hello
import typer
def main():
hello()
if __name__ == "__main__":
typer.run(main)
message.py
==
def hello():
print("hello world")
pyinstaller worked fine taking care of message.py and typer module. But
as said in my other reply it is glibc version dependent.
> You could also build an archive (e.g. `tar`, `zip`)
> and create small installer script which unpacks and installs as
> necessary. You could put the archive at the end of the script (such
> that you have a single file which gets executed to do everything).
This I will try next.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
On Thu, 9 Dec 2021 17:34:03 + Barry wrote: > > On 8 Dec 2021, at 18:27, Manfred Lotz wrote: > > > > The are many possibilities to package a Python app, and I have to > > admit I am pretty confused. > > > > Here is what I have: > > > > A Python command line app which requires some packages which are > > not in the standard library. > > > > I am on Linux and like to have an executable (perhaps a zip file > > with a shebang; whatever) which runs on different Linux systems. > > > > Different mean > > - perhaps different glibc versions > > - perhaps different Python versions > > > > In my specific case this is: > > - RedHat 8.4 with Python 3.6.8 > > - Ubuntu 20.04 LTS with Python 3.8.10 > > - and finally Fedora 33 with Python 3.9.9 > > > > > > Is this possible to do? If yes which tool would I use for this? > > Have a look at pyinstaller it knows how to collect up all your > dependancies and build a folder of files you can distribute. It’s > also got a single file mode that you might find useful. The end user > does not need to install python. > > I am looking at using it to build gui apps a macOs. > > You will have to experiment to find out if it solves you glib case > concerns. > I played with pyinstaller which worked fine. However, it builds a dynamic executable and thus it is glibc version dependent. Means, I have to build different executables for differen glibc versions. So, it seems I will have to check how executable zip archives are supposed to work. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
Hi Loris, On Wed, 08 Dec 2021 15:38:48 +0100 "Loris Bennett" wrote: > Hi Manfred, > > Manfred Lotz writes: > > > The are many possibilities to package a Python app, and I have to > > admit I am pretty confused. > > > > Here is what I have: > > > > A Python command line app which requires some packages which are > > not in the standard library. > > > > I am on Linux and like to have an executable (perhaps a zip file > > with a shebang; whatever) which runs on different Linux systems. > > > > Different mean > > - perhaps different glibc versions > > - perhaps different Python versions > > > > In my specific case this is: > > - RedHat 8.4 with Python 3.6.8 > > - Ubuntu 20.04 LTS with Python 3.8.10 > > - and finally Fedora 33 with Python 3.9.9 > > > > > > Is this possible to do? If yes which tool would I use for this? > > I use poetry[1] on CentOS 7 to handle all the dependencies and create > a wheel which I then install to a custom directory with pip3. > > You would checkout the repository with your code on the target system, > start a poetry shell using the Python version required, and then build > the wheel. From outside the poetry shell you can set PYTHONUSERBASE > and then install with pip3. You then just need to set PYTHONPATH > appropriately where ever you want to use your software. > In my case it could happen that I do not have access to the target system but wants to handover the Python app to somebody else. This person wants just to run it. > Different Python versions shouldn't be a problem. If some module > depends on a specific glibc version, then you might end up in standard > dependency-hell territory, but you can pin module versions of > dependencies in poetry, and you could also possibly use different > branches within your repository to handle that. > I try to avoid using modules which depeng on specific glibc. Although, it seems that it doesn't really help for my use case I will play with poetry to get a better understanding of its capabilities. -- Thanks a lot, Manfred -- https://mail.python.org/mailman/listinfo/python-list
How to package a Python command line app?
The are many possibilities to package a Python app, and I have to admit I am pretty confused. Here is what I have: A Python command line app which requires some packages which are not in the standard library. I am on Linux and like to have an executable (perhaps a zip file with a shebang; whatever) which runs on different Linux systems. Different mean - perhaps different glibc versions - perhaps different Python versions In my specific case this is: - RedHat 8.4 with Python 3.6.8 - Ubuntu 20.04 LTS with Python 3.8.10 - and finally Fedora 33 with Python 3.9.9 Is this possible to do? If yes which tool would I use for this? -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
On Thu, 9 Dec 2021 13:51:00 -0700 Mats Wichmann wrote: > On 12/9/21 11:35, Manfred Lotz wrote: > > > I played with pyinstaller which worked fine. However, it builds a > > dynamic executable and thus it is glibc version dependent. Means, I > > have to build different executables for differen glibc versions. > > > > So, it seems I will have to check how executable zip archives are > > supposed to work. > > > > For me at least, I'm still not sure what you are trying to accomplish. > I like to offer my command line app to some people who are not really command line geeks. Means, I don't want to have to tell them to install packages via pip and stuff like that. Simply take a file, make it executable and run it. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to package a Python command line app?
On Thu, 9 Dec 2021 21:23:58 +0100 "Peter J. Holzer" wrote: > On 2021-12-09 19:35:37 +0100, Manfred Lotz wrote: > > I played with pyinstaller which worked fine. However, it builds a > > dynamic executable and thus it is glibc version dependent. Means, I > > have to build different executables for differen glibc versions. > > Just build it for the oldest version. Wasn't aware of that. > Even if you don't, it may not > matter. I just compiled a hello world program on Ubuntu 20 and ran it > on Debian 6 (which is 10 years old). Obviusly a real program has more > opportunities to run into compatibility problems, but glibc doesn't > change all that much. > > hp > I have build an executable with pyinstaller on Ubuntu 20.04 and it didn't run on Redhat 8.4. Doing it the other way round helps indeed, i.e. building on Redhat 8.4 and then it runs on Ubuntu 20.04. Thank you. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to detect an undefined method?
On 3/27/22 18:57, Kirill Ratkin wrote:
> Hi
>
> You can get all methods of your object and check the method you want to call
> is
> there or not.
>
> |methods = [method for method in dir() if
> callable(getattr(, method))] if 'method_you_need' in methods:
> . // BR |
>
I don't understand how this may help. Assume somebody has a codebase of 15T
lines of Python code. How do you want to apply your method?
But perhaps I overlook things.
--
Manfred
> 27.03.2022 12:24, Manfred Lotz пишет:
>> Let's say I have a Python app and have used an undefined method somewhere.
>> Let
>> us further assume I have not detected it thru my tests.
>>
>> Is there a way to detect it before deploying the app? pylint doesn't notice
>> it.
>>
>>
>> Minimal example:
>>
>> #!/usr/bin/env python3
>>
>> import logging
>> from logging import Logger
>> from random import randrange
>>
>> def main():
>> """
>> Below logger.err gives
>>
>> 'Logger' object has no attribute 'err'
>> """
>>
>> logger = logging.getLogger('sample')
>> logger.setLevel(logging.DEBUG)
>> handler = logging.StreamHandler()
>> logger.addHandler(handler)
>>
>> num = randrange(0,1000)
>> if num == 0:
>> logger.err("got zero")
>> else:
>> logger.info(f'got a positive integer: {num}')
>>
>> if __name__ == "__main__":
>> main()
>>
>>
>>
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to detect an undefined method?
On 3/27/22 18:36, Skip Montanaro wrote: >> Let's say I have a Python app and have used an undefined method somewhere. >> Let >> us further assume I have not detected it thru my tests. >> >> Is there a way to detect it before deploying the app? pylint doesn't >> notice it. >> > > This is maybe not exactly what you're looking for, but writing a test to > exercise that function call (if possible) would be top of my list. Next on > my list would be to use coverage to get a good idea of what code is not > tested. Writing more test cases to reduce the number of uncovered lines > will also make it easier to manually examine the rest of your (untested) code > to find calls to missing functions or methods. > If the code base is large it might be difficult to have a test case for all things. But I agree that coverage is surely a good thing to use. -- Manfred -- https://mail.python.org/mailman/listinfo/python-list
Re: How to detect an undefined method?
On 3/27/22 18:12, Peter Otten wrote:
> On 27/03/2022 11:24, Manfred Lotz wrote:
>> Let's say I have a Python app and have used an undefined method somewhere.
>> Let
>> us further assume I have not detected it thru my tests.
>>
>> Is there a way to detect it before deploying the app? pylint doesn't notice
>> it.
>>
>>
>> Minimal example:
>>
>> #!/usr/bin/env python3
>>
>> import logging
>> from logging import Logger
>> from random import randrange
>>
>> def main():
>> """
>> Below logger.err gives
>>
>> 'Logger' object has no attribute 'err'
>> """
>>
>> logger = logging.getLogger('sample')
>> logger.setLevel(logging.DEBUG)
>> handler = logging.StreamHandler()
>> logger.addHandler(handler)
>>
>> num = randrange(0,1000)
>> if num == 0:
>> logger.err("got zero")
>> else:
>> logger.info(f'got a positive integer: {num}')
>>
>> if __name__ == "__main__":
>> main()
>
> mypy --strict will find that one. Be warned though that this is a
> slippery slope: you may end up fixing quite a few non-errors...
>
Great. I wasn't aware of --strict. Regarding 'slippery slope': I anyway would
only fix what I believe is an error.
Thanks.
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
How to detect an undefined method?
Let's say I have a Python app and have used an undefined method somewhere. Let
us further assume I have not detected it thru my tests.
Is there a way to detect it before deploying the app? pylint doesn't notice it.
Minimal example:
#!/usr/bin/env python3
import logging
from logging import Logger
from random import randrange
def main():
"""
Below logger.err gives
'Logger' object has no attribute 'err'
"""
logger = logging.getLogger('sample')
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler()
logger.addHandler(handler)
num = randrange(0,1000)
if num == 0:
logger.err("got zero")
else:
logger.info(f'got a positive integer: {num}')
if __name__ == "__main__":
main()
--
Manfred
--
https://mail.python.org/mailman/listinfo/python-list
