Re: getting rid of the recursion in __getattribute__

[Peter Otten]

On 24/05/2023 15:37, A KR wrote:

It is perfectly explained in the standards here [1] saying that:


In order to avoid infinite recursion in this method, its implementation should 
always call the base class method with the same name to access any attributes 
it needs, for example, object.__getattribute__(self, name).


Therefore, I wrote a code following what the standard says:


class Sample():
 def __init__(self):
 self.a = -10

 def __getattribute__(self, name):
 if name == 'a':
 return object.__getattribute__(self, name)

 raise AttributeError()

s = Sample()
result = s.a
print(result)

I did not fall into recursion, and the output was
-10


While this works it's not how I understand the recommended pattern. I'd
rather treat "special" attributes first and then use the
__getattribute__ method of the base class as a fallback:

>> class Demo:
def __getattribute__(self, name):
if name == "answer":
return 42
return super().__getattribute__(name)

That way your special arguments,

>>> d = Demo()
>>> d.answer
42


missing arguments

>>> d.whatever
Traceback (most recent call last):
  File "", line 1, in 
d.whatever
  File "", line 5, in __getattribute__
return super().__getattribute__(name)
AttributeError: 'Demo' object has no attribute 'whatever'

and "normal" arguments are treated as expected

>>> d.question = "What's up?"
>>> d.question
"What's up?"

Eventual "special" arguments in the superclass would also remain accessible.



However, when I try the code without deriving from a class:

class AnyClassNoRelation:
 pass

class Sample():
 def __init__(self):
 self.a = -10

 def __getattribute__(self, name):
 if name == 'a':
 return AnyClassNoRelation.__getattribute__(self, name)

 raise AttributeError()

s = Sample()

result = s.a
print(result)
and calling __getattribute__ via any class (in this example class 
AnyClassNoRelation) instead of object.__getattribute__(self, name) as the 
standard says call using the base class, I get the same output: no recursion 
and -10.

So my question:

How come this is possible (having the same output without using the base 
class's __getattribute__? Although the standards clearly states that 
__getattribute__ should be called from the base class.



AnyClassNoRelation does not override __getattribute__, so

>>> AnyClassNoRelation.__getattribute__ is object.__getattribute__
True


There is no sanity check whether a method that you call explicitly is
actually in an object's inheritance tree,

>>> class NoRelation:
def __getattribute__(self, name):
return name.upper()


>>> class Demo:
def __getattribute__(self, name):
return "<{}>".format(NoRelation.__getattribute__(self, name))


>>> Demo().some_arg
''

but the only purpose I can imagine of actually calling "someone else's"
method is to confuse the reader...


In order to avoid infinite recursion in this method, its implementation should 
always call the base class method with the same name to access any attributes 
it needs, for example, object.__getattribute__(self, name).


Literally, I can call __getattribute__ with anyclass (except Sample cause it 
will be infinite recursion) I define and it works just fine. Could you explain 
me why that happens?



--
https://mail.python.org/mailman/listinfo/python-list

Re: Question regarding unexpected behavior in using __enter__ method

[Peter Otten]

On 21/04/2023 00:44, Lorenzo Catoni wrote:

Dear Python Mailing List members,

I am writing to seek your assistance in understanding an unexpected
behavior that I encountered while using the __enter__ method. I have
provided a code snippet below to illustrate the problem:

```

class X:

... __enter__ = int
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...

x

0
```
As you can see, the __enter__ method does not throw any exceptions and
returns the output of "int()" correctly. However, one would normally expect
the input parameter "self" to be passed to the function.

On the other hand, when I implemented a custom function in place of the
__enter__ method, I encountered the following TypeError:

```

def myint(*a, **kw):

... return int(*a, **kw)
...

class X:

... __enter__ = myint
... __exit__ = lambda *_: None
...

with X() as x:

... pass
...
Traceback (most recent call last):
   File "", line 1, in 
   File "", line 2, in myint
TypeError: int() argument must be a string, a bytes-like object or a real
number, not 'X'
```
Here, the TypeError occurred because "self" was passed as an input
parameter to "myint". Can someone explain why this unexpected behavior
occurs only in the latter case?


Cameron is right, it's the descriptor protocol. Technically

inst.attr

invokes attr.__get__(...) if it exists:

>>> class A:
def __get__(self, *args): return args


>>> class B: pass

>>> class X:
a = A()
b = B()


>>> x = X()
>>> x.b
<__main__.B object at 0x02C2E388>
>>> x.a
(<__main__.X object at 0x02C2E280>, )

Python functions support the descriptor protocol

>>> hasattr(lambda: None, "__get__")
True

while builtin functions don't:

>>> hasattr(ord, "__get__")
False


I'm unsure whether to regard int as a class or or function, but as there
is no __get__

>>> hasattr(int, "__get__")
False

it behaves like builtin functions in this case.

--
https://mail.python.org/mailman/listinfo/python-list

Re: When is logging.getLogger(__name__) needed?

[Peter Otten]

On 31/03/2023 15:01, Loris Bennett wrote:

Hi,

In my top level program file, main.py, I have

   def main_function():

   parser = argparse.ArgumentParser(description="my prog")

   ...

   args = parser.parse_args()
   config = configparser.ConfigParser()

   if args.config_file is None:
   config_file = DEFAULT_CONFIG_FILE
   else:
   config_file = args.config_file

   config.read(config_file)

   logging.config.fileConfig(fname=config_file)
   logger = logging.getLogger(__name__)

   do_some_stuff()

   my_class_instance = myprog.MyClass()

   def do_some_stuff():

   logger.info("Doing stuff")

This does not work, because 'logger' is not known in the function
'do_some_stuff'.

However, if in 'my_prog/my_class.py' I have

   class MyClass:

   def __init__(self):

   logger.debug("created instance of MyClass")

this 'just works'.


Take another look at your code -- you'll probably find


   logger = logging.getLogger(__name__)


on the module level in my_class.py.


to 'do_some_stuff', but why is this necessary in this case but not in
the class?


Your problem has nothing to do with logging -- it's about visibility
("scope") of names:

>>> def use_name():
print(name)


>>> def define_name():
name = "Loris"


>>> use_name()
Traceback (most recent call last):
  File "", line 1, in 
use_name()
  File "", line 2, in use_name
print(name)
NameError: name 'name' is not defined

Binding (=assigning to) a name inside a function makes it local to that
function. If you want a global (module-level) name you have to say so:

>>> def define_name():
global name
name = "Peter"


>>> define_name()
>>> use_name()
Peter

--
https://mail.python.org/mailman/listinfo/python-list

Re: Custom help format for a choice argparse argument

[Peter Otten]

On 27/01/2023 21:31, Ivan "Rambius" Ivanov wrote:

Hello,

I am developing a script that accepts a time zone as an option. The
time zone can be any from pytz.all_timezones. I have

def main():
 parser = argparse.ArgumentParser()
 parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
 args = parser.parse_args()
 print(args)
 print(f"Specified timezone: {args.zone}")

It works, but when I run it with the -h option it dumps all entries in
pytz.all_timezones. I would like to modify the help format for just
-z|--zone option. I read the docs about HelpFormatter and argparse.py
and I ended up with

class CustomHelpFormatter(argparse.HelpFormatter):
 def _metavar_formatter(self, action, default_metavar):
 if action.dest == 'zone':
 result = 'zone from pytz.all_timezones'
 def format(tuple_size):
 if isinstance(result, tuple):
 return result
 else:
 return (result, ) * tuple_size
 return format
 else:
 return super(CustomHelpFormatter,
self)._metavar_formatter(action, default_metavar)


def main():
 parser = argparse.ArgumentParser(formatter_class=CustomHelpFormatter)
 parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
 args = parser.parse_args()
 print(args)
 print(f"Specified timezone: {args.zone}")

This works, but is there a more elegant way to achieve it?


It may be sufficient to specify a metavar:

>>> import argparse
>>> p = argparse.ArgumentParser()
>>> p.add_argument("--foo", choices="alpha beta gamma".split(),
metavar="")
[...]
>>> p.parse_args(["-h"])
usage: [-h] [--foo ]

optional arguments:
  -h, --helpshow this help message and exit
  --foo 

While that helps with --help it doesn't always prevent the choices list
from being spelt out:

>>> p.parse_args(["--foo", "whatever"])
usage: [-h] [--foo ]
: error: argument --foo: invalid choice: 'whatever' (choose from
'alpha', 'beta', 'gamma')

--
https://mail.python.org/mailman/listinfo/python-list

Re: Keeping a list of records with named fields that can be updated

[Peter Otten]

On 18/12/2022 16:44, songbird wrote:

Peter Otten wrote:
...

While I think what you need is a database instead of the collection of
csv files the way to alter namedtuples is to create  a new one:


from collections import namedtuple
Row = namedtuple("Row", "foo bar baz")
row = Row(1, 2, 3)
row._replace(bar=42)

Row(foo=1, bar=42, baz=3)


   namedtuple is easier to use as that will use the csv and
csvreader and create the records without me having to do any
conversion or direct handling myself.  it's all automagically
done.  my initial version works, but i'd like it to be a bit
more elegant and handle descriptions it hasn't seen before
in a more robust manner.



An alternative would be dataclasses where basic usage is just as easy:


from dataclasses import make_dataclass
Row = make_dataclass("Row", "foo bar baz".split())
row = Row(1, 2, 3)
row

Row(foo=1, bar=2, baz=3)

row.bar = 42
row

Row(foo=1, bar=42, baz=3)


   i do like that i can directly reference each field in a
dataclass and not have to specify a _replace for each change.

   is there an easy way to convert from namedtuple to dataclass?
i can see there is a _asdict converter, but don't really like
how that turns out as then i have to do a bunch of:
 rec['fieldname'] = blah

rec.fieldname is much easier to understand.


I recommend that you use a dataclass /instead/ of a namedtuple, not
both. However, for a dataclass with the same fields in the same order as
in your namedtuple the conversion is trivial:

Create compatible namedtuple and dataclass types:

>>> NTRow = namedtuple("NTRow", ["alpha", "beta", "gamma"])
>>> DCRow = make_dataclass("DCRow", NTRow._fields)

Build the namedtuple:

>>> ntrow = NTRow(1, "two", 3.0)
>>> ntrow
NTRow(alpha=1, beta='two', gamma=3.0)

Convert to dataclass:

>>> dcrow = DCRow(*ntrow)
>>> dcrow
DCRow(alpha=1, beta='two', gamma=3.0)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Keeping a list of records with named fields that can be updated

[Peter Otten]

On 17/12/2022 20:45, Albert-Jan Roskam wrote:

On Dec 15, 2022 10:21, Peter Otten <__pete...@web.de> wrote:

  >>> from collections import namedtuple
  >>> Row = namedtuple("Row", "foo bar baz")
  >>> row = Row(1, 2, 3)
  >>> row._replace(bar=42)
  Row(foo=1, bar=42, baz=3)


Ahh, I always thought these are undocumented methods, but: "In addition to
the methods inherited from tuples, named tuples support three additional
methods and two attributes. To prevent conflicts with field names, the
method and attribute names start with an underscore."

https://docs.python.org/3/library/collections.html#collections.somenamedtuple._make


I've read somewhere that Raymond Hettinger regrets the naming and now
would favour a trailing underscore to avoid name conflicts.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Keeping a list of records with named fields that can be updated

[Peter Otten]

On 14/12/2022 19:50, songbird wrote:


   I'm relatively new to python but not new to programming in general.

   The program domain is accounting and keeping track of stock trades and other 
related information (dates, cash accounts, interest, dividends, transfers of 
funds, etc.)

   Assume that all data is CSV format.  There are multiple files.

   Assume there is a coherent starting point and that all data is in order.

   Assume each line contains a description.  The description determines what 
the line is.  The number of fields in the line does not change within the data 
file but it may happen that later lines in other files may be different other 
than the fact that they all must contain a description.

   All descriptions are deterministic (none are recursive or referencing things 
from the future).  All things referenced in the description which do not 
already exist are added to a list (or perhaps more than one in a few cases) and 
may contain some basic information (the date, how many and for how much, or a 
total amount or a fee or ...)  If the field of the line isn't a number it is 
either a symbol or a description.

   A default action is simply to keep most parts of the line and to adjust any 
totals of a previously seen description that matches by whatever amounts are on 
the line.  The key is the description.

   I've already written one program based upon the files I already have which 
works but what happens is that new descriptions are added (new accounts, new 
stocks, etc.) and I don't want to have to write new code manually every time a 
description changes.

   I started using named tuples (it works for reading in the files and 
accessing the fields) but I cannot update those so I need to use something else 
to give me the list of unique descriptions and fields that I need to update.  
I've not gotten beyond that yet as I'm still learning.

   Suggestions?

   Thanks!  :)


While I think what you need is a database instead of the collection of
csv files the way to alter namedtuples is to create  a new one:

>>> from collections import namedtuple
>>> Row = namedtuple("Row", "foo bar baz")
>>> row = Row(1, 2, 3)
>>> row._replace(bar=42)
Row(foo=1, bar=42, baz=3)

An alternative would be dataclasses where basic usage is just as easy:

>>> from dataclasses import make_dataclass
>>> Row = make_dataclass("Row", "foo bar baz".split())
>>> row = Row(1, 2, 3)
>>> row
Row(foo=1, bar=2, baz=3)
>>> row.bar = 42
>>> row
Row(foo=1, bar=42, baz=3)


--
https://mail.python.org/mailman/listinfo/python-list

Re: Top level of a recursive function

[Peter Otten]

On 13/12/2022 15:46, Michael F. Stemper wrote:

It's easy enough -- in fact necessary -- to handle the bottom
level of a function differently than the levels above it. What
about the case where you want to handle something differently
in the top level than in lower levels? Is there any way to tell
from within a function that it wasn't invoked by itself?

I've come up with a hack to support different processing, by
use of an extra argument, as shown in this simplified example:

def fred(cf,toplevel=True):
   x = cf[0]
   if len(cf)>1:
     if toplevel:
   return x + fred(cf[1:],False)
     else:
   return "(" + x + fred(cf[1:],False) + ")"
   else:
     if toplevel:
   return x
     else:
   return "(" + x + ")"

Aside from being ugly, this lets the caller diddle with "toplevel",
which shouldn't really be externally modifiable.

Are there better ways to do this?


For adepts of functional programming the above is a "fold right"
operation, first hit for "foldr in python":

https://burgaud.com/foldl-foldr-python

With that

>>> from functools import reduce
>>> def foldr(func, items):
return reduce(lambda x, y: func(y, x), items[::-1])

your problem reduces ;) to

>>> foldr("{0}({1})".format, [1,2,3,4])
'1(2(3(4)))'

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?

[Peter Otten]

On 08/12/2022 02:17, Jach Feng wrote:

Peter Otten 在 2022年12月8日 星期四清晨5:17:59 [UTC+8] 的信中寫道：

On 07/12/2022 03:23, Jach Feng wrote:

s0 = r'\x0a'
At this moment it was done by

def to1byte(matchobj):
return chr(int('0x' + matchobj.group(1), 16))
s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)

But, is it that difficult on doing this simple thing?

import codecs
codecs.decode(r"\x68\x65\x6c\x6c\x6f\x0a", "unicode-escape")

'hello\n'

Thank you. What I really want to handle is to any r'\xdd'. The r'\x0a' is for 
example. Sorry, didn't describe it clearly:-)


Hm, codecs.decode() does work for arbitrary escapes. It will produce the
same result for r"\xdd"-type raw strings where d is in the range 0...F.
It will also convert other escapes like

>>> codecs.decode(r"\t", "unicode-escape")
'\t'
>>> codecs.decode(r"\u5728", "unicode-escape")
'在'

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to convert a raw string r'\xdd' to '\xdd' more gracefully?

[Peter Otten]

On 07/12/2022 03:23, Jach Feng wrote:

s0 = r'\x0a'
At this moment it was done by

 def to1byte(matchobj):
 return chr(int('0x' + matchobj.group(1), 16))
 s1 = re.sub(r'\\x([0-9a-fA-F]{2})', to1byte, s0)

But, is it that difficult on doing this simple thing?


>>> import codecs
>>> codecs.decode(r"\x68\x65\x6c\x6c\x6f\x0a", "unicode-escape")
'hello\n'

--
https://mail.python.org/mailman/listinfo/python-list

Re: typing: property/setter and lists?

[Peter Otten]

On 03/11/2022 04:24, Paulo da Silva wrote:

Hi!

And a typing problem again!!!
___
class C:
 def __init__(self):
     self.__foos=5*[0]

 @property
 def foos(self) -> list[int]:
     return self.__foos

 @foos.setter
 def foos(self,v: int):
     self.__foos=[v for __i in self.__foos]

c=C()
c.foos=5
print(c.foos)
___

mypy gives the following error:
error: Incompatible types in assignment (expression has type "int",
variable has type "List[int]")

How do I turn around this?


Quoting https://github.com/python/mypy/issues/3004:

"""
gvanrossum commented on Mar 15, 2017
IIRC we debated a similar issues when descriptors were added and decided
that it's a perverse style that we just won't support. If you really
have this you can add # type: ignore.
"""

--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: A typing question

[Peter Otten]

On 30/10/2022 14:37, Peter J. Holzer wrote:

On 2022-10-30 09:23:27 -0400, Thomas Passin wrote:

On 10/30/2022 6:26 AM, Peter J. Holzer wrote:

On 2022-10-29 23:59:44 +0100, Paulo da Silva wrote:

The funny thing is that if I replace foos by Foos it works because it gets
known by the initial initialization :-) !


from typing import List, Optional

class GLOBALS:
  Foos: Optional[Foos]=None

[...]

class Foos:


That seems like a bug to me. What is the «Foos» in «Optional[Foos]»
referring to?

If it's the class attribute «Foos» then that's not a type and even if
its type is inferred that's not the same as «Optional[it's type]», or is
it?

If it's referring to the global symbol «Foos» (i.e. the class defined
later) that hasn't been defined yet, so it shouldn't work (or
alternatively, if forward references are allowed it should always work).


Quoting a forward-referenced type is the way to use one.  Unquoted types
need to have been declared already.


Yes. I was referring to the code as written. Why does that work? I don't
think it should.


For me it makes sense. I think mypy should refrain from trying to figure
out order of execution. If the above is disallowed, how about

if random.randrange(2):
class A: pass

class B(A): pass

?

One interesting consequence of that policy -- take the whole scope
instead of the most recent appearance of a name is that

class A: pass
class A: pass

won't compile.

While I didn't expect that I think I like it ;)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Typing: Is there a "cast operator"?

[Peter Otten]

On 24/10/2022 05:19, Chris Angelico wrote:

On Mon, 24 Oct 2022 at 14:15, Dan Stromberg  wrote:

I've found that mypy understands simple assert statements.

So if you:
if f is not None:
 assert f is not None
 os.write(f, ...)

You might be in good shape.


Why can't it simply understand the if statement?


Could it be that this specific problem is fixed in current mypy?
I get

$ type .\tmp.py
import os

f = None
if f is None:
f = os.open("tmp.txt", os.O_RDWR|os.O_CREAT)
os.write(f, b"yadda")
$ mypy tmp.py
Success: no issues found in 1 source file

My attempt to verify that

if name is None: ...

is recognized:

$ type .\tmp2.py
import os
import random

f = None
if random.randrange(2):
f = os.open("tmp.txt", os.O_RDWR|os.O_CREAT)
os.write(f, b"yadda")
$ mypy tmp2.py
tmp2.py:7: error: Argument 1 to "write" has incompatible type
"Optional[int]"; expected "int"
Found 1 error in 1 file (checked 1 source file)
$ mypy --version
mypy 0.982 (compiled: yes)
$

I'm not a fan of

coddling a type system like this. The entire point of type checking is
to help you find bugs more efficiently, so if you have to repeat
yourself every time you do these kinds of checks just so that mypy is
satisfied, that's counter-productive (case in point: what happens if
you say "if fn is not None: assert f is not None"? Now you've
introduced a bug just to deal with the type system).

ChrisA


--
https://mail.python.org/mailman/listinfo/python-list

Re: Mutating an HTML file with BeautifulSoup

[Peter Otten]

On 22/08/2022 05:30, Chris Angelico wrote:

On Mon, 22 Aug 2022 at 10:04, Buck Evan  wrote:


I've had much success doing round trips through the lxml.html parser.

https://lxml.de/lxmlhtml.html

I ditched bs for lxml long ago and never regretted it.

If you find that you have a bunch of invalid html that lxml inadvertently 
"fixes", I would recommend adding a stutter-step to your project: perform a 
noop roundtrip thru lxml on all files. I'd then analyze any diff by progressively 
excluding changes via `grep -vP`.
Unless I'm mistaken, all such changes should fall into no more than a dozen 
groups.



Will this round-trip mutate every single file and reorder the tag
attributes? Because I really don't want to manually eyeball all those
changes.


Most certainly not. Reordering is a bs4 feature that is governed by a
formatter. You can easily prevent that attributes are reorderd:

>>> import bs4
>>> soup = bs4.BeautifulSoup("")
>>> soup

>>> class Formatter(bs4.formatter.HTMLFormatter):
def attributes(self, tag):
return [] if tag.attrs is None else list(tag.attrs.items())

>>> soup.decode(formatter=Formatter())
''

Blank space is probably removed by the underlying html parser.
It might be possible to make bs4 instantiate the lxml.html.HTMLParser
with remove_blank_text=False, but I didn't try hard enough ;)

That said, for my humble html scraping needs I have ditched bs4 in favor
of lxml and its xpath capabilities.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Object in List : how?

[Peter Otten]

On 25/07/2022 02:47, Khairil Sitanggang wrote:

Regarding your comment : "
*However, usually object creation and initialization iscombined by allowing
arguments to the initializer:*" , so which one of the two classes Node1,
Node2 below is more common in practice? Option 2, I guess.
Thanks,


# option 1:
class Node1:
 def __init__(self, a):
 self.a = a
 self.b = self.calculation()

 def calculation(self):
 r = self.a + 10
 return r

# option 2:
class Node2:
 def __init__(self, a, b):
 self.a = a
 self.b = b

 self.b = self.calculation()

 def calculation(self):
 r = self.a + 10
 return r

nd1 = Node1(10)
nd2 = Node2(10, 0) # 0 is dummy, will be overwritten by the call to
calculation()


An attribute that can be calculated from other attributes should never
be modified through other means. Some day you may want b to become
something else, write, for example,

node = Node2(10, "twenty")

and because by then you have forgotten about the calculation() call end
up with a buggy script. But manually invoking the calculation() method
is also bug prone. You have to remember to repeat it every time you
change a:

node = Node1(10)
assert node.b == 20  # OK

node.a = 20
assert node.b == 30 # fails, a and b are out of sync.

The solution Python has to offer is called "property". Properties in
their simplest form are calculated read-only attributes, i. e. when you
write

print(node.b)

under the hood node.a + 10 is calculated. Here's how to change Node1 to
turn b into such a property:

class Node3a:
def __init__(self, a):
self.a = a
def calculation(self):
return self.a + 10
b = property(calculation)

node = Node3a(42)
print(node.b)  # 52

node.a = 1
print(node.b)  # 11

Often you are not interested in exposing both the calculation() method
and the property. For cases when you only want to access the property
Python provides a way to define the property with a "decorator":

class Node3b:
def __init__(self, a):
self.a = a
@property
def b(self):
return self.a + 10

When you compare the two classes you can see that I

(1) renamed calculation() to b() and

(2) replaced

def b(self): ...
b = property(b)

with

@property
def b(self): ...

thus avoiding the repetitons of the name.

Are there any disadvantages to properties?

What I presented as an advantage, that the value of the attribute is
recalculated every time the attribute is accessed, may sometimes become
a disadvantage, e. g. when it takes a very long time to calculate. In
most cases that should not be a problem, though.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Object in List : how?

[Peter Otten]

On 23/07/2022 06:28, Khairil Sitanggang wrote:

Hello Expert:

I just started using python. Below is a simple code.  I was trying to check
if, say, NO1 is not in the NODELIST[:].NO
How can I achieve this purpose?

Regards,
-Irfan


class Node:
 def __init__(self):
 self.NO = 0
 self.A = 20

NODE = Node()
NODELIST = []

NODE.NO = 10
NODELIST.append(NODE)

NODE.NO = 20


This just overwrites the attribute; the previous value 10 is lost.


NODELIST.append(NODE)

NODE.NO = 30


This againoverwrites the attribute; the previous value 20 is lost.


NODELIST.append(NODE)



You are three times appending the *same* node to the list.
To create a new node you need to invoke the initializer:

[I'm following a common convention and use lowercase names in my examples]

nodelist = []

# first node
node = Node()
node.no = 10
nodelist.append(node)

# second node
node = Node()  # this is crucial
node.no = 20
nodelist.append(node)

... and so on. However, usually object creation and initialization is
combined by allowing arguments to the initializer:

class Node:
def __init__(self, no, a):
self.no = no
self.a = a

nodelist = []
for no in [10, 20, 30]:
nodelist.append(Node(no, 20))


NO1 = 20
if NO1 not in NODELIST[:].NO  ???


You are checking if the list contains an item with a specific attribute
value, so you cannot use the nodelist directly, you need an intermediate
list that contains the attribute values:

no1 = 20
nos = [node.no for node in nodelist]
if no1 not in nos:
print("not found")

There's one disadvantage to this approach. If the node list is huge
another huge list with the attribute values is built even though the
first item in the nodelist may already have the searched-for attribute
value. To avoid the overhead you could write a function:

def contains_no(nodes, no):
for node in nodes:
if node.no == no:
return True
return False

if not contains_no(nodelist, 20):
print("not found")

But Python has something more elegant, a kind of /lazy/ /list/ called
"generator expression" where each item is calculated on demand. With
that you can write

if 20 not in (node.no for node in nodelist):
print("not found")

and your script will stop inspecting further nodes as soon as a matching
node is found.
--
https://mail.python.org/mailman/listinfo/python-list

Re: list indices must be integers or slices, not str

[Peter Otten]

On 20/07/2022 11:37, Chris Angelico wrote:

On Wed, 20 Jul 2022 at 18:34, Frank Millman  wrote:


Hi all

C:\Users\E7280>python
Python 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
  >>>
  >>> x = list(range(10))
  >>>
  >>> '{x[1]}'.format(**vars())
'1'
  >>>
  >>> '{x[-1]}'.format(**vars())
Traceback (most recent call last):
File "", line 1, in 
TypeError: list indices must be integers or slices, not str
  >>>

Can anyone explain this error? It seems that a negative index is deemed
to be a string in this case.



Yeah, that does seem a little odd. What you're seeing is the same as
this phenomenon:


"{x[1]} {x[spam]}".format(x={1: 42, "spam": "ham"})

'42 ham'

"{x[1]} {x[spam]}".format(x={"1": 42, "spam": "ham"})

Traceback (most recent call last):
   File "", line 1, in 
KeyError: 1

But I can't find it documented anywhere that digits-only means
numeric.



I found

https://peps.python.org/pep-3101/

"""
PEP 3101 – Advanced String Formatting
...
An example of the ‘getitem’ syntax:

"My name is {0[name]}".format(dict(name='Fred'))

It should be noted that the use of ‘getitem’ within a format string is
much more limited than its conventional usage. In the above example, the
string ‘name’ really is the literal string ‘name’, not a variable named
‘name’. The rules for parsing an item key are very simple. If it starts
with a digit, then it is treated as a number, otherwise it is used as a
string.

Because keys are not quote-delimited, it is not possible to specify
arbitrary dictionary keys (e.g., the strings “10” or “:-]”) from within
a format string.
"""
--
https://mail.python.org/mailman/listinfo/python-list

Re: Creating lambdas inside generator expression

[Peter Otten]

On 29/06/2022 23:17, Chris Angelico wrote:

On Thu, 30 Jun 2022 at 02:49, Johannes Bauer  wrote:

But now consider what happens when we create the lambdas inside a list
comprehension (in my original I used a generator expresison, but the
result is the same). Can you guess what happens when we create conds
like this?

conds = [ lambda msg: msg.hascode(z) for z in ("foo", "bar") ]

I certainly could not. Here's what it outputs:

Check for bar
False
Check for bar
False

I.e., the iteration variable "z" somehow gets bound inside the lambda
not by its value, but by its reference. All checks therefore refence
only the last variable.



Yep, that is the nature of closures. (Side point: This isn't actually
a generator expression, it's a list comprehension; current versions of
Python treat them broadly the same way, but there was previously a
difference in the way scoping worked.) What you're seeing is a
consequence of the way that closures work, and it is a very good thing
most of the time :)

The usual way to "snapshot" a variable is what you showed in your
followup: a default argument value.

def f(..., z=z):
 ... z has been snapshot

(As others have pointed out, this isn't unique to lambdas; any
function will behave that way.)

Antoon offered another variant, but written as a pair of lambda
functions, it's a little hard to see what's going on. Here's the same
technique written as a factory function:

def does_it_have(z):
 return lambda msg: msg.hascode(z)

conds = [does_it_have(z) for z in ("foo", "bar")]

Written like this, it's clear that the variable z in the comprehension
is completely different from the one inside does_it_have(), and they
could have different names if you wanted to. This is a fairly clean
way to snapshot too, and has the advantage that it doesn't pretend
that the function takes an extra parameter.


While I'd go with Chris' suggestion there are two other options:
functools.partial() and operator.methodcaller().

Example:

>>> class Msg:
def __init__(self, msg):
self.msg = msg
def hascode(self, code):
return code in self.msg


>>> conds = [partial(lambda z, msg: msg.hascode(z), z) for z in ("foo",
"bar")]
>>> [cond(Msg("barbaz")) for cond in conds]
[False, True]
>>> conds = [methodcaller("hascode", z) for z in ("foo", "bar")]
>>> [cond(Msg("barbaz")) for cond in conds]
[False, True]


--
https://mail.python.org/mailman/listinfo/python-list

Re: python Store text file in mangodb

[Peter Otten]

On 12/06/2022 14:40, Ayesha Tassaduq wrote:

Hi i am trying to store a text file into MongoDB but i got the error .
"failing because no such method exists." % self.__name.split(".")[-1]
TypeError: 'Collection' object is not callable. If you meant to call the 
'insert' method on a 'Collection' object it is failing because no such method 
exists.

Can anyone please tell what is wrong here
i also tried to do it with insert_one  and insert_many
but when i try to do it with insert_many it shows error
   raise TypeError("documents must be a non-empty list")
TypeError: documents must be a non-empty list


Read the error messages carefully:

(1) "...If you meant to call the 'insert' method on a 'Collection'
object it is failing because no such method exists."

It may be a bit unfortunate that attributes spring into existence when
you try to access them, but you want an  "insert" method, and the error
message warns you that no such method exists. You can run your script in
idle and then type

>>> collections.insert

to see what it actually is.

(2) "...documents must be a non-empty list"

The insert_many() method expects a non-empty list of documents Example:

collection.insert_many([text_file_doc])

You don't provide an error message for insert_one(), and indeed it
should work where you tried insert():

collection.insert(text_file_doc)



from pymongo import MongoClient
client = MongoClient()
db = client.test_database  # use a database called "test_database"
collection = db.files   # and inside that DB, a collection called "files"

f = open('hashes.txt')  # open a file

# build a document to be inserted
text_file_doc = {"file_name": "hashes.txt"}
# insert the contents into the "file" collection
collection.insert(text_file_doc)

File names Hshes.txt has follown=ing data


You are not yet at the point where you are using the file or its
contents, so the file object and the file's contents could be omitted.
Generally it is a good idea

- to make your script as short as possible as long as it still produces
the error. In the process you will often be able to fix the problem
yourself.

- always provide the tracebacks using cut-and-paste. That is often
sufficient to diagnose and fix the problem.


Hash 1: 39331a6a2ea1cf31a5014b2a7c9e8dfad82df0b0666e81ce04cf8173cc5aed

Hash 2: 0e0ff63b7e5e872b9ea2f0d604b5d5afd6ba05665e52246fa321ead5b79c00ad

Hash 3: 89241ce841704508be1d0b76c478c9575ec8a7ec8be46742fd5acb0dc72787f3

Hash 4: 80283cb08f91b415aae04bcada0da1ca3e37bbe971ae821116b4d29008970bdb


--
https://mail.python.org/mailman/listinfo/python-list

Re: PYLAUNCH_DEBUG not printing info

[Peter Otten]

On 09/06/2022 00:53, Richard David wrote:

Why am I not getting debug output on my windows 10 machine:

C:\temp>\Windows\py.exe -0
  -V:3.11 *Python 3.11 (64-bit)
  -V:3.10  Python 3.10 (64-bit)

C:\temp>set PYLAUNCH_DEBUG=1

C:\temp>\Windows\py.exe
Python 3.11.0b3 (main, Jun  1 2022, 13:29:14) [MSC v.1932 64 bit (AMD64)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.

^Z




Looks like the variable is now called PYLAUNCHER_DEBUG:

https://docs.python.org/3.11/using/windows.html#diagnostics
--
https://mail.python.org/mailman/listinfo/python-list

Re: PYLAUNCH_DEBUG not printing info

[Peter Otten]

On 09/06/2022 00:53, Richard David wrote:

Why am I not getting debug output on my windows 10 machine:

C:\temp>\Windows\py.exe -0
  -V:3.11 *Python 3.11 (64-bit)
  -V:3.10  Python 3.10 (64-bit)

C:\temp>set PYLAUNCH_DEBUG=1

C:\temp>\Windows\py.exe
Python 3.11.0b3 (main, Jun  1 2022, 13:29:14) [MSC v.1932 64 bit (AMD64)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.

^Z




Does getenv() confirm that the variable is set?

>>> import os
>>> os.getenv("PYLAUNCH_DEBUG")
'1'
--
https://mail.python.org/mailman/listinfo/python-list

Re: Filtering XArray Datasets?

[Peter Otten]

On 07/06/2022 00:28, Israel Brewster wrote:

I have some large (>100GB) datasets loaded into memory in a two-dimensional (X 
and Y) NumPy array backed XArray dataset. At one point I want to filter the data 
using a boolean array created by performing a boolean operation on the dataset 
that is, I want to filter the dataset for all points with a longitude value 
greater than, say, 50 and less than 60, just to give an example (hopefully that 
all makes sense?).

Currently I am doing this by creating a boolean array (data[‘latitude’]>50, for 
example), and then applying that boolean array to the dataset using .where(), with 
drop=True. This appears to work, but has two issues:

1) It’s slow. On my large datasets, applying where can take several minutes 
(vs. just seconds to use a boolean array to index a similarly sized numpy array)
2) It uses large amounts of memory (which is REALLY a problem when the array is 
already using 100GB+)

What it looks like is that values corresponding to True in the boolean array 
are copied to a new XArray object, thereby potentially doubling memory usage 
until it is complete, at which point the original object can be dropped, 
thereby freeing the memory.

Is there any solution for these issues? Some way to do an in-place filtering?


Can XArray-s be sorted, resized  in-place? If so, you can sort by
longitude <= 50, search the index of the first row with longitude <= 50
and then resize the array.

(If the order of rows matters the sort algorithme has to be stable)
--
https://mail.python.org/mailman/listinfo/python-list

Re: oop issue

[Peter Otten]

On 23/05/2022 22:54, Tola Oj wrote:

i just finished learning oop as a beginner and trying to practice with it
but i ran into this typeerror issue, help please.

Traceback (most recent call last):
   File
"c:\Users\ojomo\OneDrive\Desktop\myexcel\oop_learn.py\myExperiment.py\mainMain.py",
line 36, in 
 print(invest_crypto.client_list)
TypeError: invest_crypto.__repr__() missing 1 required positional argument:
'self'



 @staticmethod
 def __repr__(self):
 return f"('{self.name}', '{self.surname}', '{self.amount_Deposited}',
'{self.amount_to_transfer}')"


What are you trying to achieve with the staticmethod decorator?
--
https://mail.python.org/mailman/listinfo/python-list

Re: EAFP

[Peter Otten]

On 13/05/2022 18:37, bryangan41 wrote:

Is the following LBYL:foo = 123if foo < 200:    do()If so, how to change to 
EAFP?Thanks!Sent from Samsung tablet.


The distinction between look-before-you-leap and
easier-to-ask-forgiveness-than-permission is weaker than yo might expect.

When you write

filename = ...
if exists(filename):
with open(filename) as instream:
# do stuff
else:
# fallback

there are two checks for the file's existence, one explicit, and one
implicitly inside open() -- and worse, the first, explicit, check is
unreliable because between exists() and open() there is a small delay
that may be sufficient to create or delete the file. Therefore the
recommended (EAFP) version of the above is

filename = ...
try:
with open(filename) as instrem:
# do stuff
except FileNotFoundError
# fallback

or just

with open(filename) as instream:
# do stuff

if there is no meaningful fallback.

Regarding your example code, whether you can write an EAFP version for

if foo < 200:
do()

depends on the contents of do(). If do() fails in a well-defined way,
let's say by raising a FooAbove199 exception you can write just

do()

or, if you need a fallback

try:
do()
except FooAbove199:
# fallback

Note that if you change do() from

do():
# do stuff

to

def do():
if foo < 200:
# do stuff
else:
raise FooAbove199(f"Invalid {foo=!r}")

the

do()

invocation becomes EAFP even though you are actually performing the same
test as before.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Tuple unpacking inside lambda expressions

[Peter Otten]

On 20/04/2022 13:01, Sam Ezeh wrote:

I went back to the code recently and I remembered what the problem was.

I was using multiprocessing.Pool.pmap which takes a callable (the
lambda here) so I wasn't able to use comprehensions or starmap

Is there anything for situations like these?


Hm, I don't see pmap, but there is a starmap():

https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.starmap
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to detect an undefined method?

[Peter Otten]

On 27/03/2022 11:24, Manfred Lotz wrote:

Let's say I have a Python app and have used an undefined method somewhere. Let
us further assume I have not detected it thru my tests.

Is there a way to detect it before deploying the app? pylint doesn't notice it.


Minimal example:

#!/usr/bin/env python3

import logging
from logging import Logger
from random import randrange

def main():
 """
 Below logger.err gives

 'Logger' object has no attribute 'err'
 """

 logger = logging.getLogger('sample')
 logger.setLevel(logging.DEBUG)
 handler = logging.StreamHandler()
 logger.addHandler(handler)

 num = randrange(0,1000)
 if num == 0:
 logger.err("got zero")
 else:
 logger.info(f'got a positive integer: {num}')

if __name__ == "__main__":
 main()


mypy --strict will find that one. Be warned though that this is a
slippery slope: you may end up fixing quite a few non-errors...

--
https://mail.python.org/mailman/listinfo/python-list

Re: All permutations from 2 lists

[Peter Otten]

On 02/03/2022 01:32, Rob Cliffe via Python-list wrote:


itertools.product returns an iterator (or iterable, I'm not sure of the
correct technical term).


There's a simple test:

iter(x) is x --> True  # iterator
iter(x) is x --> False  # iterable


So:

>>> from itertools import product
>>> p = product("ab", [1, 2])
>>> iter(p) is p  # iterator
True
>>> items = [1, 2]  # iterable
>>> iter(items) is items
False

Another interesting property of (finite) iterators/iterables

list(iterable) == list(iterable) --> Generally True, but not guaranteed.

a = list(iterator)  # whatever
b = list(iterator)  # [] (*)

(*) Kill the coder if that doesn't hold ;)
--
https://mail.python.org/mailman/listinfo/python-list

Re: One-liner to merge lists?

[Peter Otten]

On 27/02/2022 17:28, Chris Angelico wrote:

On Mon, 28 Feb 2022 at 03:24, MRAB  wrote:


On 2022-02-27 08:51, Barry Scott wrote:




On 22 Feb 2022, at 09:30, Chris Angelico  wrote:

On Tue, 22 Feb 2022 at 20:24, Frank Millman mailto:fr...@chagford.com>> wrote:


Hi all

I think this should be a simple one-liner, but I cannot figure it out.

I have a dictionary with a number of keys, where each value is a single
list -


d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}


I want to combine all values into a single list -


ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']


I can do this -


a = []
for v in d.values():

...   a.extend(v)
...

a

['aaa', 'bbb', 'ccc', 'fff', 'ggg']

I can also do this -


from itertools import chain
a = list(chain(*d.values()))
a

['aaa', 'bbb', 'ccc', 'fff', 'ggg']




Is there a simpler way?



itertools.chain is a good option, as it scales well to arbitrary
numbers of lists (and you're guaranteed to iterate over them all just
once as you construct the list). But if you know that the lists aren't
too large or too numerous, here's another method that works:


sum(d.values(), [])

['aaa', 'bbb', 'ccc', 'fff', 'ggg']

It's simply adding all the lists together, though you have to tell it
that you don't want a numeric summation.


If you code is performance sensitive do not use sum() as it creates lots of tmp 
list that are deleted.

I have an outstanding ticket at work to replace all use of sum() on lists as 
when we profiled it
stands out as a slow operation. We have a lots of list of list that we need to 
flatten.


I think that 'sum' uses '__add__' but not '__iadd__'.

If it copied the given value, if present, and then used '__iadd__', if
present, wouldn't that speed it up?


It's hardly relevant. If you care about speed, use chain(), like
everyone's been saying all along :)


Not everyone. I believe (*) that __iadd__ is faster, and again, you can
spell it

functools.reduce(operator.iconcat, list_of_lists, [])

(*) Actual measurements left as an exercise ;)

--
https://mail.python.org/mailman/listinfo/python-list

Re: One-liner to merge lists?

[Peter Otten]

On 22/02/2022 10:44, Frank Millman wrote:

On 2022-02-22 11:30 AM, Chris Angelico wrote:

On Tue, 22 Feb 2022 at 20:24, Frank Millman  wrote:


Hi all

I think this should be a simple one-liner, but I cannot figure it out.

I have a dictionary with a number of keys, where each value is a single
list -

  >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}

I want to combine all values into a single list -

  >>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']

I can do this -

  >>> a = []
  >>> for v in d.values():
...   a.extend(v)
...
  >>> a
['aaa', 'bbb', 'ccc', 'fff', 'ggg']

I can also do this -

  >>> from itertools import chain
  >>> a = list(chain(*d.values()))
  >>> a
['aaa', 'bbb', 'ccc', 'fff', 'ggg']
  >>>

Is there a simpler way?



itertools.chain is a good option, as it scales well to arbitrary
numbers of lists (and you're guaranteed to iterate over them all just
once as you construct the list). But if you know that the lists aren't
too large or too numerous, here's another method that works:


sum(d.values(), [])

['aaa', 'bbb', 'ccc', 'fff', 'ggg']

It's simply adding all the lists together, though you have to tell it
that you don't want a numeric summation.



Thanks, that is neat.

However, I did see this -

 >>> help(sum)
Help on built-in function sum in module builtins:

sum(iterable, /, start=0)
     Return the sum of a 'start' value (default: 0) plus an iterable of
numbers

     When the iterable is empty, return the start value.
     This function is intended specifically for use with numeric values
and may reject non-numeric types.
 >>>

So it seems that it is not recommended.


If you don't like the idea of 'adding' strings you can 'concat'enate:

>>> items = [[1,2,3], [4,5], [6]]
>>> functools.reduce(operator.concat, items)
[1, 2, 3, 4, 5, 6]
>>> functools.reduce(operator.iconcat, items, [])
[1, 2, 3, 4, 5, 6]

The latter is the functional way to spell your for... extend() loop.
Don't forget to provide the initial value in that case lest you modify
the input:

>> functools.reduce(operator.iconcat, items)  # wrong
[1, 2, 3, 4, 5, 6]
>>> items
[[1, 2, 3, 4, 5, 6], [4, 5], [6]]  # oops


--
https://mail.python.org/mailman/listinfo/python-list

Re: Multiple inheritance using super() in parent classes

[Peter Otten]

On 10/02/2022 09:20, Igor Basko wrote:

Hi everyone,
This is my first question here. Hope to get some clarification.
Basically this question is about multiple inheritance and the usage of
super().__init__ in parent
classes.

So I have two classes that inherit from the same base class.
For example class B and class C inherit from A:
class A:
   def __init__(self, arg1):
 pass

class B(A):
   def __init__(self, arg2):
 super().__init__(arg2)

class C(A):
   def __init__(self, arg1, arg2):
 super().__init__(arg2)

Now I would like to create a new class D that inherits from B and C.
One note, D is the only class that I am "allowed" to change. A, B and C are
provided to me as is from an external package.
class D(B, C):
   def __init__(self):
 B.__init__(self, 'arg1')
 C.__init__(self, 'arg1', 'arg2')

When I initialize D I get a TypeError.
TypeError: __init__() missing 1 required positional argument: 'arg2'
I get it from the invocation of super().__init__(arg2) inside the B class.

As I understand it, the super() inside B tries to call the __init__ of
class C,
because of the multiple inheritance and the MRO that is constructed.
But when B was implemented it wasn't aware of C and I assume,
B shouldn't be aware of C in any case.


Even when you call the initializers explicitly you pass self, and that's
probably where the MRO is taken from. You can tweak that MRO by changing
the order of the parent classes:

class D(C, B): ...

However, this will call B.__init__() twice; therefore I'd stick to super():

class D(C, B):
def __init__(self):
super().__init__("arg1", "arg2")


It gives me the feeling that I'm trying to implement some bad practice
here, but I'm not sure why.


I have the feeling that your feeling is right ;)


I would also like to hear your suggestions if there is a way to circumvent
it. Maybe by the approach described here:
https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
wrapping B and C in some Adapter class.


A quick glance at that page suggests that the adapter translates the
base class into an attribute -- which is what I would have suggested,
too. Using has-a instead of is-a relations is generally something to
consider seriously before jumping into multiple inheritance. The details
may depend on the actual use case which is hidden behind your A, B, C,
and D abstraction...

--
https://mail.python.org/mailman/listinfo/python-list

Re: The code version of python -i

[Peter Otten]

On 15/09/2021 15:39, Abdur-Rahmaan Janhangeer wrote:

Greetings,

If i have a file name flower.py and i add x = 1 in it.
When i run python -i flower.py i get a shell




If type x i get 1

x

1

The values are auto injected.

How do i start a shell by code with values already injected? Thanks

Kind Regards,


I tried

import code

x = 42
code.interact(local=globals())

but didn't bother to read the documentation at

https://docs.python.org/3/library/code.html

and thus do not know what the limitations might be.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Trouble propagating logging configuration

[Peter Otten]

On 01/09/2021 13:48, Loris Bennett wrote:

"Dieter Maurer"  writes:


Loris Bennett wrote at 2021-8-31 15:25 +0200:

I am having difficulty getting the my logging configuration passed on
to imported modules.

My initial structure was as follows:

  $ tree blorp/
  blorp/
  |-- blorp
  |   |-- __init__.py
  |   |-- bar.py
  |   |-- foo.py
  |   `-- main.py
  `-- pyproject.toml

whereby the logging configuration is done in main.py.

After thinking about it, I decided maybe the inheritance wasn't working
because main.py is in the same directory as the other files.


Should you speak about Python's `logging` module, then
the "inheritance" does not depend on the source layout.
Instead, it is based on the hierarchy of dotted names.
It is completely up to you which dotted names you are using
in your `getLogger` calls.


Yes, but to quote from 
https://docs.python.org/3.6/howto/logging.html#logging-basic-tutorial:

   A good convention to use when naming loggers is to use a module-level
   logger, in each module which uses logging, named as follows:

 logger = logging.getLogger(__name__)

   This means that logger names track the package/module hierarchy, and
   it’s intuitively obvious where events are logged just from the logger
   name.

so in this case the source layout is relevant, isn't it?


In most cases you will only add handlers to the root logger. If you want 
special treatment of a specific logger you need to know its name 
including its package, i. e. log_test.first, not first. If you have the 
log_test directory in your path and use


import first

instead of

import log_test.first

then not only the __name__ will be wrong/unexpected, relative imports 
will fail, too.

In other words, this is generally a bad idea.


Furthermore, the place of the configuration (and where in the
code it is activated) is completely irrelevant for the "inheritance".


OK, so one issue is that I was getting confused by the *order* in which
modules are being called.  If I have two modules, 'foo' and 'bar', in
the same directory, configure the logging just in 'foo' and then call


   foo.some_method()
   bar.some_method()

then both methods will be logged.   If I do

   bar.some_method()
   foo.some_method()

then only the method in 'foo' will be logged.

However, I still have the following problem.  With the structure

   $ tree .
   .
   |-- log_test
   |   |-- __init__.py
   |   |-- first.py
   |   `-- second.py
   |-- pyproject.toml
   |-- README.rst
   |-- run.py
   `-- tests
   |-- __init__.py
   |-- config
   `-- test_log_test.py

I have __name__ variables as follows:

   __file__: /home/loris/log_test/log_test/first.py, __name__: log_test.first
   __file__: /home/loris/log_test/log_test/second.py, __name__: log_test.second
   __file__: ./run.py, __name__: __main__

If I have

   [loggers]
   keys=root,main,log_test

in my logging configuration and initialise  the logging in run.py with

   logging.config.fileConfig("/home/loris/log_test/tests/config")
   logger = logging.getLogger()

or

   logging.config.fileConfig("/home/loris/log_test/tests/config")
   logger = logging.getLogger("log_test")

then only calls in 'run.py' are logged.

I can obviously initialise the logging within the subordinate package,
i.e. in 'log_test/__init__.py', but that seems wrong to me.

So what is the correct way to initialise logging from a top-level script
such that logging is activated in all modules requested in the logging
configuration?


For details, read the Python documentation for the `logging` module.


If they were sufficient, I wouldn't need the newsgroup :-)

Thanks for the help,


Perhaps you can start with logging.basicConfig() in run.py. If the 
imported modules contain module-level logging messages you have to put 
the import statements after the basicConfig() invocation.


You should see that simple changes like setting the logging level affect 
messages from all loggers. If that's not enough to get the granularity 
you want give your loggers fixed names


# in test_log/first.py
logger = logging.getLogger("test_log.first")

or make sure that __name__ is what you expect

# in test_log/first.py
assert __name__ == "test_log.first"

during the debugging phase.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Request for argmax(list) and argmin(list)

[Peter Otten]

On 01/09/2021 06:25, ABCCDE921 wrote:

I dont want to import numpy

argmax(list)
returns index of (left most) max element



>>> import operator
>>> second = operator.itemgetter(1)
>>> def argmax(values):
return max(enumerate(values), key=second)[0]

>>> argmax([1, 2, 3, 0])
2


  argmin(list)
returns index of (left most) min element


This is left as an exercise;)



--
https://mail.python.org/mailman/listinfo/python-list

Re: on writing a while loop for rolling two dice

[Peter Otten]

On 30/08/2021 06:17, dn via Python-list wrote:


OTOH the simulation of rolling n-number of dice, as would happen in the
real-world, would be broken by making the computer's algorithm more
efficient (rolling until the first non-equal value is 'found'). Does
that mean the realism of the model dies?


You've got to ask your dietician...

(I'm sure everyone agrees that I should stop here. And stop I will)

--
https://mail.python.org/mailman/listinfo/python-list

Re: on writing a while loop for rolling two dice

[Peter Otten]

On 30/08/2021 15:50, Chris Angelico wrote:


def how_many_times():
 return next((count, rolls) for count, rolls in
enumerate(iter(roll, None)) if len(Counter(rolls)) == 1)



That's certainly the most Counter-intuitive version so far;)



Do I get bonus points for it being a one-liner that doesn't fit in
eighty characters?


Nah, but you'll get an honorable mention when you run it through 
pycodestyle without line break...


--
https://mail.python.org/mailman/listinfo/python-list

Re: code to initialize a sequence

[Peter Otten]

On 29/08/2021 20:44, joseph pareti wrote:

In the code attached below, the A-variant is from somebody else who knows
Python better than I. But I do not like to just use any code without having
a grasp, specifically the line in* bold*, so I wrote the B-variant which
gives the same results. The C-variant is identical to A and is there for
verification: after resetting the seed I expect the same sequence. The
D-variant is closer to the way I code, and it does not work.


So you do you want us to debug the _EXPLICIT version?
The assignment

>  X[i] = [randint(0, n_unique-1)]

creates a list with one element and turns it into an item in the list X.
You don't want  a list, you want the numerical value, the 
straight-forward way to achieve that being


X[i] = randint(0, n_unique-1)

An alternative is to assign to a slice

X[i:i+1] = [randint(...)]

but that would only make sense if the right-hand-side list weren't 
created in the line and ditched immediately afterwards.





import random
from random import randint, seed

def generate_sequence(length, n_unique):
*return [randint(0, n_unique-1) for k in range(length)]*


The above is the most pythonic of the three versions. Once you 
understand how for loops with a list.append() are turned into 
comprehensions it will be easy to write and read that style. Definitely 
worth learning and adopting.




def generate_sequence_JP(length, n_unique):
LI = []
for k in range(length):
  LI.append(randint(0, n_unique-1))
return(LI)



This is also fine and often used when the loop body is a bit more 
complex, but



def generate_sequence_EXPLICIT(length, n_unique):
X =[None] * length
   for i in range(length):
 X[i] = [randint(0, n_unique-1)]
return X


this is garbage even when it works, usually indicative of premature 
optimization.


Random (yeah!) remark: Python uses half-open intervals (i. e. intervals 
that include the lower bound, but not the upper bound) almost 
everywhere. randint() is one exception.

Personally I prefer its conformist sister randrange(); with that
randint(0, n_unique-1)
becomes
randrange(n_unique)


#
# MAIN PROGRAM
#
random.seed(2)
A = generate_sequence(4, 10 )
random.seed(2)
B = generate_sequence_JP(4, 10)
random.seed(2)
C = generate_sequence(4, 10 )
random.seed(2)
D = generate_sequence_EXPLICIT(4, 10 )
print(A)
print(type(A))
print('-')
print(B)
print(type(B))
print('-')
print(C)
print(type(C))
print('-')
print(D)
print(type(D))


Regards,
Joseph Pareti - Artificial Intelligence consultant
Joseph Pareti's AI Consulting Services
https://www.joepareti54-ai.com/
cell +49 1520 1600 209
cell +39 339 797 0644




--
https://mail.python.org/mailman/listinfo/python-list

Re: on writing a while loop for rolling two dice

[Peter Otten]

On 29/08/2021 12:13, dn via Python-list wrote:

On 29/08/2021 20.06, Peter Otten wrote:
...

OK, maybe a bit complicated... but does it pay off if you want to
generalize?


def roll_die(faces):

 while True: yield random.randrange(1, 1 + faces)


def hmt(faces, dies):

 for c, d in enumerate(zip(*[roll_die(faces)]*dies), 1):
     if len(set(d)) == 1: return c, d



Curiosity:
why not add dies as a parameter of roll_die()?


Dunno. Maybe because I've "always" [1] wanted a version of 
random.randrange() that generates values indefinitely. It would need to 
check its arguments only once, thus leading to some extra



Efficiency:
- wonder how max( d ) == min( d ) compares for speed with the set() type
constructor?


I did the simplest thing, speed was not a consideration. If it is, and 
dies (sorry for that) is large I'd try


first = d[0]
all(x == first for x in d)  # don't mind one duplicate test

For smaller numbers of dice I'd unpack (first, *rest) inside the for 
loop. But it's a trade-off, you' have to measure if/when it's better to 
go through the whole tuple in C.




- alternately len( d ) < 2?
- or len( d ) - 1 coerced to a boolean by the if?
- how much more efficient is any of this (clever thinking!) than the
OP's basic, simpler, and thus more readable, form?


It really isn't efficiency, it's a (misled?) sense of aesthetics where 
I've come to prefer


- for-loops over while, even when I end up with both to get the desired for

- enumerate() over an explicit counter even though there is the extra 
unpack, and you still need to initialize the counter in the general case:


for i, item in enumerate([]): pass
print(f"There are {i+1} items in the list.")  # Oops


English language 'treachery':
- one die
- multiple dice


You might have inferred that I knew (or had looked up) the singular of 
dice, so this is but a momentary lapse of reason. It hurts me more than 
you, trust me. Not as much, as going on record with confusing they're 
and their, but still ;)



(probably not followed in US-English (can't recall), particularly on
computers running the Hollywood Operating System).


I've come to the conclusion that International English is hopelessly and 
inevitably broken. That's the price native speakers have to pay for 
having they're (oops, I did it again!) language used as lingua franca.



Continuous Education:
Thanks for the reminder that enumerate() can be seeded with a "start" value!


[1] I think I've suggested reimplementing the whole module in terms of 
generators -- can't find the post though.


--
https://mail.python.org/mailman/listinfo/python-list

Re: on writing a while loop for rolling two dice

[Peter Otten]

On 28/08/2021 14:00, Hope Rouselle wrote:


def how_many_times():
   x, y = 0, 1
   c = 0
   while x != y:
 c = c + 1
 x, y = roll()
   return c, (x, y)
--8<---cut here---end--->8---

Why am I unhappy?  I'm wish I could confine x, y to the while loop.  The
introduction of ``x, y = 0, 1'' must feel like a trick to a novice.  How
would you write this?  Thank you!


I'd probably hide the while loop under the rug:

>>> import random
>>> def roll_die():
while True: yield random.randrange(1, 7)


Then:

>>> def hmt():
for c, (x, y) in enumerate(zip(roll_die(), roll_die()), 1):
if x == y:
return c, (x, y)


>>> hmt()
(1, (2, 2))
>>> hmt()
(4, (4, 4))
>>> hmt()
(1, (5, 5))


OK, maybe a bit complicated... but does it pay off if you want to 
generalize?


>>> def roll_die(faces):
while True: yield random.randrange(1, 1 + faces)

>>> def hmt(faces, dies):
for c, d in enumerate(zip(*[roll_die(faces)]*dies), 1):
if len(set(d)) == 1: return c, d


>>> hmt(10, 1)
(1, (2,))
>>> hmt(10, 2)
(3, (10, 10))
>>> hmt(10, 3)
(250, (5, 5, 5))
>>> hmt(1, 10)
(1, (1, 1, 1, 1, 1, 1, 1, 1, 1, 1))

You decide :)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pandas: Multiple CSVs in one Zip file

[Peter Otten]

On 26/08/2021 09:09, Abdur-Rahmaan Janhangeer wrote:


Cannot read one file in a zip file if the zip file contains multiple files.

This example does not work https://www.py4u.net/discuss/203494 as Pandas
shows a ValueError: Multiple files found in ZIP file. Only one file per ZIP.

If the Zip file has one file, fine else i cannot find a way. Thanks


You may have introduced an error when you translated the snippet to your 
use case. What went wrong is hard to guess unless you provide the exact 
failing code.


Pandas can read a single zipped file out of the box and the accepted 
answer in your indirect link to


https://stackoverflow.com/questions/61971040/python-extract-csv-files-from-multiple-zip-files-and-combine-the-data

shows how to deal with zip files containing multiple files.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Recoding Categorical to Numerical

[Peter Otten]

On 14/08/2021 00:33, John Griner wrote:

Hello, and thanks in advance,

  I am trying to recode categorical variable to numeric.  Despite using
lambda, If else, dummy-recoding, and so forth, what I get is from ONE
column that has 4 variables (City, Town, Suburb, Rural), I get FOUR columns:



localeDummy_City  localeDummy_Town  localeDummy_Suburb
localeDummy_Rural   locale_recode



with the corresponding numeric variable.



What I want is the recode to have ONE column that has the numeric
conversion.



For instance:


local_recode

2

4

4

6

2

8

6

2

8

2

2

4

6

4

8

and so forth, where I have set City to 2, and Town to 4, etc.


Again, thanks, John


My crystal ball says you want

import pandas

df = pandas.DataFrame(
[
[("City", "Suburb")],
[("Town", "City", "Suburb")],
[("Rural",)]
],
columns=["before"]
)

flags = dict(
City=1,
Town=2,
Suburb=4,
Rural=8
)

df["after"] = df["before"].apply(
lambda names: sum(flags[name] for name in set(names))
)

print(df)

If that's not it show us your failing code, preferably as a small 
self-contained script that also generates the required input data. Use 
cut and paste, and include it into the message body as attachments are 
usually removed.



--
https://mail.python.org/mailman/listinfo/python-list

Re: pip doesn't work

[Peter Otten]

On 13/08/2021 06:49, Ridit wrote:

So, whoever gets this, when I try to download packages using pip, it shows
errors saying "'pip' is not recognized as an internal or external command,
operable program or batch file." Is there a way to solve this? I tried
modifying, even repairing three times, but still it doesn't work. If you
have a solution, thank you.


Try invoking it with

py -m pip ...

instead of just

pip ...

--
https://mail.python.org/mailman/listinfo/python-list

Re: Generate a Google Translate API token through the Socks5 proxy using gtoken.py

[Peter Otten]

On 29/07/2021 17:43, Dennis Lee Bieber wrote:

On Thu, 29 Jul 2021 15:45:26 +0200, Peter Otten <__pete...@web.de>
declaimed the following:


On 28/07/2021 18:40, Dennis Lee Bieber wrote:

On Wed, 28 Jul 2021 09:04:40 +0200, Peter Otten <__pete...@web.de>
declaimed the following:



Perhaps it has something to do with the X-No-Archive flag set by Dennis?


According to my properties page in Agent, I've turned that off except
if the message I'm replying to has it set.


I'm yet to see an archived post by you -- maybe that setting is
overridden somewhere.


And naturally, once I found the legacy (per persona) custom setting, I
failed to edit the main persona. Hopefully this time it's set...


https://mail.python.org/pipermail/python-list/2021-July/902975.html

You are now officially archived ;)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Generate a Google Translate API token through the Socks5 proxy using gtoken.py

[Peter Otten]

On 28/07/2021 18:40, Dennis Lee Bieber wrote:

On Wed, 28 Jul 2021 09:04:40 +0200, Peter Otten <__pete...@web.de>
declaimed the following:



Perhaps it has something to do with the X-No-Archive flag set by Dennis?


According to my properties page in Agent, I've turned that off except
if the message I'm replying to has it set.


I'm yet to see an archived post by you -- maybe that setting is 
overridden somewhere.


--
https://mail.python.org/mailman/listinfo/python-list

Re: [Errno 2] No such file or directory:

[Peter Otten]

On 28/07/2021 18:09, joseph pareti wrote:

The following code fails as shown in the title:






*import subprocesscmd = 'ls -l
/media/joepareti54/Elements/x/finance-2020/AI/Listen_attend_spell/VCTK-Corpus/wav48
| awk "{print  $9 }"'process = subprocess.Popen([cmd],


As a quick fix try

process = subprocess.Popen(
cmd # no list
shell=True,
... # other args as above
)


  stdout=subprocess.PIPE, stderr=subprocess.PIPE)stdout, stderr =
process.communicate()print('stdout ',stdout)print('stderr ',stderr)*



Traceback (most recent call last):
   File "PreProcess_1a.py", line 3, in 
 process = subprocess.Popen([cmd],  stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
   File "/home/joepareti54/anaconda3/lib/python3.8/subprocess.py", line 854,
in __init__
 self._execute_child(args, executable, preexec_fn, close_fds,
   File "/home/joepareti54/anaconda3/lib/python3.8/subprocess.py", line
1702, in _execute_child
 raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ls -l
/media/joepareti54/Elements/x/finance-2020/AI/Listen_attend_spell/VCTK-Corpus/wav48
| awk "{print  $9




--
https://mail.python.org/mailman/listinfo/python-list

Re: Generate a Google Translate API token through the Socks5 proxy using gtoken.py

[Peter Otten]

On 28/07/2021 07:32, Cameron Simpson wrote:

On 27Jul2021 19:24, Hongyi Zhao  wrote:

On Wednesday, July 28, 2021 at 7:25:27 AM UTC+8, cameron...@gmail.com wrote:

Just to follow on a bit to Dennis:


But I can't any reply from Dennis in this issue.


Odd, because I replied to his reply to you :-)


Perhaps it has something to do with the X-No-Archive flag set by Dennis?

--
https://mail.python.org/mailman/listinfo/python-list

Re: Searching pypi.org, is there an 'advanced search'?

[Peter Otten]

On 18/07/2021 03:40, MRAB wrote:

On 2021-07-17 13:01, Chris Green wrote:



pypi.org is a wonderful resource but its size now demands a better
search engine.


There's always Google. I find that the search terms:

     "google contacts" pypi

finds some results.


With a small modification

"google contacts" site:pypi.org

the number of results goes down to 25 (of which 13 are shown by default).

--
https://mail.python.org/mailman/listinfo/python-list

Re: Correct traceback for multiline chain of method calling

[Peter Otten]

On 09/07/2021 17:29, Артем Комендантян wrote:

Hello!

There is a code https://pastebin.com/0NLsHuLa.
It has a multiline chain of method calling, when some method fails. In
python3.7 it fails in a row which corresponds to the failing method, in
python3.9 it corresponds to the very first line.

Another similar example is https://pastebin.com/2P9snnMn
The error is on the first line for older pythons too.

I propose to have a line with the method name in traceback if this method
fails.

I develop some library when it is very popular among users to declare some
operations with such multiline chains. Also I want to keep correct
traceback for each operation because the first line is not very informative
when some method occurred more than once in this chain.

Can this improvement be done? Maybe anybody has any other suggestions on
how to get the correct line in traceback right now?


If I understand you correctly there's hope; the upcoming Python 3.10 
will show the failing method call:


Python 3.10.0b3 (tags/v3.10.0b3:865714a, Jun 17 2021, 20:19:11) [MSC 
v.1929 32 bit (Intel)] on win32

Type "help", "copyright", "credits" or "license()" for more information.
class A:
def chain(self): return self
def fail(self): 1/0

(A()
 .chain()
 .chain()
 .fail()
 .chain()
 )
Traceback (most recent call last):
  File "", line 4, in 
.fail()
  File "", line 3, in fail
def fail(self): 1/0
ZeroDivisionError: division by zero


--
https://mail.python.org/mailman/listinfo/python-list

Re: Unfindable module: code

[Peter Otten]

On 02/07/2021 11:44, Chris Angelico wrote:

I've just spent half an hour trying to figure out how to mess with the
Python REPL (specifically, how to implement a line-by-line interactive
interpreter within a larger app). It's rather hard to find it, but the
key module is "code".

https://docs.python.org/3/library/code.html

(How did I end up finding it? By searching the CPython source code for
"ps1", since interactive mode looks at sys.ps1/sys.ps2 for its
prompts.)


I would probably have done that too, but for "interpreter".

However, googling for "python interactive interpreter module" has the 
code documentation as its second hit and


https://docs.python.org/3/library/custominterp.html

as its third.


In the module index, it is listed thus:

Custom Python Interpreters
* code — Interpreter base classes
* codeop — Compile Python code

While this isn't *wrong*, per se, it does hide the fact that this is
where the REPL can be found. IMO it would be helpful to say that in
the summary, but I'm not sure what would be good wording.

What do people think of calling it "Interactive interpreter and REPL
implementation"?


To be honest, I think the "unfindable" part are the module names; I 
don't see a big difference between your suggested and the current 
description.


--
https://mail.python.org/mailman/listinfo/python-list

Re: creating raw AWS log event

[Peter Otten]

On 23/06/2021 19:42, Larry Martell wrote:

When an AWS cloudwatch event is passed to a consumer it looks like this:

{
 "awslogs": {
  "data": "ewogICAgIm1l..."
  }
}

To get the actual message I do this:

def _decode(data):
 compressed_payload = b64decode(data)
 json_payload = zlib.decompress(compressed_payload, 16+zlib.MAX_WBITS)
 return json.loads(json_payload)

message = _decode(json.dumps(event['awslogs']['data']))

This returns the log message as a string.

For my unit tests I need to reverse this - given a message as a string
I want to generate the compressed, encoded event structure.

I have not been able to get this to work. I have this:

message  = b'test message'
compressed= zlib.compress(message)
event['awslogs']['data'] = str(compressed)

message = _decode(json.dumps(event['awslogs']['data']))
Traceback (most recent call last):
   File "", line 1, in 
   File "", line 3, in _decode
zlib.error: Error -3 while decompressing data: incorrect header check

Anyone see how to make this work?


The json/bas64 parts are not involved in the problem:

>>> zlib.decompress(zlib.compress(b"foo"), 16 + zlib.MAX_WBITS)
Traceback (most recent call last):
  File "", line 1, in 
zlib.decompress(zlib.compress(b"foo"), 16 + zlib.MAX_WBITS)
zlib.error: Error -3 while decompressing data: incorrect header check

whereas:

>>> zlib.decompress(zlib.compress(b"foo"))
b'foo'

Unfortunately compress() doesn't accept the flags you seem to require.
However, reading around a bit in the zlib docs turns up the compressobj 
which does. So


>>> def mycompress(data):
obj = zlib.compressobj(wbits=16 + zlib.MAX_WBITS)
result = obj.compress(data)
result += obj.flush()
return result

>>> zlib.decompress(mycompress(b"foo"), 16 + zlib.MAX_WBITS)
b'foo'



--
https://mail.python.org/mailman/listinfo/python-list

Re: Is there a way to get the following result in Python?

[Peter Otten]

On 12/06/2021 04:02, Jach Feng wrote:


def foo():

... # do something
...

a = []
for i in range(3):

... a.append(foo())
...

a

[]


The most natural way to achieve something similar is to replace append() 
with extend():


>>> def foo(): return ()

>>> a = []
>>> for i in range(3):
a.extend(foo())


>>> a
[]



--
https://mail.python.org/mailman/listinfo/python-list

Re: Neither pdb or print() displays the bug

[Peter Otten]

On 01/06/2021 23:32, Rich Shepard wrote:

On Tue, 1 Jun 2021, Ethan Furman wrote:


Well, you only had two logging statements in that code -- logging is like
print: if you want to see it, you have to call it:


Ethan,

Got it, thanks.

I believe


Do you have unit tests? Those are an excellent tool to ensure that the 
components of an application work as expected and that those components 
have well-defined interfaces. Debugging a failing unittest is usually 
easier than to debug a complex application.
While I don't know if there is a convenient way to test the GUI 
everything else should run reliably *before* connecting it with the GUI.


my problem is with the datasource module. I'm focused on 
making it

work (using logging to confirm that it is doing so). Will report results
when they're available.

Regards,

Rich



--
https://mail.python.org/mailman/listinfo/python-list

Re: Functions as Enum member values

[Peter Otten]

On 31/05/2021 17:57, Colin McPhail via Python-list wrote:

Hi,

According to the enum module's documentation an Enum-based enumeration's 
members can have values of any type:

"Member values can be anything: int, str, etc.."


You didn't read the fineprint ;)

"""
The rules for what is allowed are as follows: names that start and end 
with a single underscore are reserved by enum and cannot be used; all 
other attributes defined within an enumeration will become members of 
this enumeration, with the exception of special methods (__str__(), 
__add__(), etc.), descriptors (methods are also descriptors), and 
variable names listed in _ignore_



Functions written in Python are descriptors and therefore cannot be 
used. Builtin functions aren't, leading to the following somewhat 
surprising difference:


>>> def foo(self): print("foo")

>>> class A(Enum):
x = foo  # foo.__get__ exists -> foo is a descriptor
 # the enum library assumes you are adding a
 # method to your
 # class, not an enumeration value.

y = sum  # no sum.__get__ attribute -> function
 # treated as an enumeration value.

>>> list(A)
[>]
>>> A.x

>>> A.y
>
>>> A.y.x()
foo


I defined one with functions as member values. This seemed to work as long as the functions 
were defined ahead of the enumeration's definition. However I then noticed that my 
enumeration class was a bit weird: even although I could reference the enumeration's 
members individually by name I couldn't iterate through the members in the normal fashion - 
it seemed to have no members. Also, calling type() on a member gave the result  instead of the expected .

I am using Python 3.9.5 from python.org on macOS 11.4. Below is a small test 
script and its output. Are my expectations wrong or is there a problem with 
Enum?

Regards,
Colin

---
from enum import Enum

def connect_impl():
 print("message from connect_impl()")

def help_impl():
 print("message from help_impl()")

class Command1(Enum):
 CONNECT = 1
 HELP = 2

class Command2(Enum):
 CONNECT = connect_impl
 HELP = help_impl

if __name__ == "__main__":

 def print_enum(cls, mbr):
 print(f"{cls.__name__}\n  type(): {type(Command1)}")
 print(f"  type of enum member: {type(mbr)}")
 print(f"  number of members: {len(cls)}")
 print("  enum members:")
 if len(cls) > 0:
 for c in cls:
 print(f"{c}")
 else:
 print("")
 print(f"  dir(): {dir(cls)}")

 member_1 = Command1.HELP
 print_enum(Command1, member_1)
 print()

 member_2 = Command2.HELP
 print_enum(Command2, member_2)
 print()
 print("call Command2 member: ", end="")
 member_2()
 print()

---
(qt6) % python3 try_enum.py
Command1
   type(): 
   type of enum member: 
   number of members: 2
   enum members:
 Command1.CONNECT
 Command1.HELP
   dir(): ['CONNECT', 'HELP', '__class__', '__doc__', '__members__', 
'__module__']

Command2
   type(): 
   type of enum member: 
   number of members: 0
   enum members:
 
   dir(): ['__class__', '__doc__', '__members__', '__module__']

call Command2 member: message from help_impl()

(qt6) %
---




--
https://mail.python.org/mailman/listinfo/python-list

Re: Applying winpdb_reborn

[Peter Otten]

On 29/05/2021 01:12, Cameron Simpson wrote:

On 28May2021 14:49, Rich Shepard  wrote:

On Fri, 28 May 2021, Dennis Lee Bieber wrote:

It's apparently looking for some environment variable based upon the
code at
https://www.google.com/url?sa=t=j==s=web==2ahUKEwjfpYmk3-zwAhULI6wKHSPjAFIQFnoECAUQAA=http%3A%2F%2Fphault.rhul.ac.uk%2Fpy%2F_spe%2Fplugins%2Fwinpdb%2Frpdb2.py=AOvVaw12BuzlEMVXrEuOFLoQLpFX


Thanks, Dennis. It looked like an environment variable but I hadn't seen
that with the python2 winpdb.


I'm concerned by the NameError, not in keeping with the fact that there
does seem to be a global (module level) variable of the right name.


I'll add that to ~/.bash_profile and see what happens.


The NameError suggests that this won't work. Maybe you're module install
is an old version? 


This seems to be the problem. There was a bugfix in November 2020:

https://github.com/bluebird75/winpdb/commit/215712d75cf89b0678d563237746be647d5f25e7

--
https://mail.python.org/mailman/listinfo/python-list

exit() builtin, was Re: imaplib: is this really so unwieldy?

[Peter Otten]

On 26/05/2021 01:02, Cameron Simpson wrote:

On 25May2021 15:53, Dennis Lee Bieber  wrote:

On Tue, 25 May 2021 19:21:39 +0200, hw  declaimed the
following:

Oh ok, it seemed to be fine.  Would it be the right way to do it with
sys.exit()?  Having to import another library just to end a program
might not be ideal.


I've never had to use sys. for exit...

C:\Users\Wulfraed>python
Python ActivePython 3.8.2 (ActiveState Software Inc.) based on
on win32
Type "help", "copyright", "credits" or "license" for more information.

exit()




I have learned a new thing today.

Regardless, hw didn't call it, just named it :-)


exit() is inserted into the built-ins by site.py. This means it may not 
be available:


PS D:\> py -c "exit('bye ')"
bye
PS D:\> py -S -c "exit('bye ')"
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'exit' is not defined

I have no idea if this is of any practical relevance...

--
https://mail.python.org/mailman/listinfo/python-list

Shadowing, was Re: learning python ...

[Peter Otten]

On 25/05/2021 05:20, hw wrote:


We're talking about many different things. If it's simply "num = ..."
followed by "num = ...", then it's not a new variable or anything,
it's simply rebinding the same name. But when you do "int = ...", it's
shadowing the builtin name.


Why?  And how is that "shadowing"?

What if I wanted to re-define the built-in thing?



When you write

foo

in a module the name "foo" is looked up in the module's global 
namespace. If it's not found it is looked up in builtins. If that lookup 
fails a NameError exception is raised.


>>> import builtins
>>> builtins.foo = "built-in foo"
>>> foo
'built-in foo'
>>> foo = "module-global-foo"  # at this point builtins.foo is shadowed
>>> foo
'module-global-foo'
>>> del foo # delete the global to make the built-in visible again:
>>> foo
'built-in foo'

That mechanism allows newbies who don't know the builtins to write

list = [1, 2, 3]

without affecting other modules they may use. It also allows old scripts 
that were written when a builtin name did not yet exist to run without 
error.


The problem you ran into, using a name in two roles

float = float("1.2")

could be circumvented by writing

float = builtins.float("1.2")

but most of the time it is more convenient to think of builtins as names 
that are predefined in your current module and act accordingly.


As you see redefining a builtin is as easy as importing builtins and 
setting the respective attribute


>>> builtins.float = int
>>> float(1.23)
1

--
https://mail.python.org/mailman/listinfo/python-list

Re: learning python ...

[Peter Otten]

On 23/05/2021 06:37, hw wrote:


Hi,

I'm starting to learn python and have made a little example program 
following a tutorial[1] I'm attaching.


Running it, I'm getting:


Traceback (most recent call last):
   File "[...]/hworld.py", line 18, in 
     print(isinstance(int, float))
TypeError: isinstance() arg 2 must be a type or tuple of types


I would understand to get an error message in line 5 but not in 18.  Is 
this a bug or a feature?


It is a bug in your code (which you don't provide). Did you assign some 
value to float, e. g.:


>>> float = 42.0
>>> isinstance(int, float)
Traceback (most recent call last):
  File "", line 1, in 
isinstance(int, float)
TypeError: isinstance() arg 2 must be a type or tuple of types

If you do not shadow the built-in you should get

>>> isinstance(int, float)
False

--
https://mail.python.org/mailman/listinfo/python-list

Instantiating abstract classes

[Peter Otten]

Usually an abstract class cannot be instantiated:

>>> from abc import ABCMeta, abstractmethod
>>> from fractions import Fraction
>>> class A(Fraction, metaclass=ABCMeta):
@abstractmethod
def frobnicate(self): pass


>>> A()
Traceback (most recent call last):
  File "", line 1, in 
A()
  File "C:\Program Files\Python39-32\lib\fractions.py", line 93, in __new__
self = super(Fraction, cls).__new__(cls)
TypeError: Can't instantiate abstract class A with abstract method 
frobnicate



However, if I derive from a builtin class that mechanism doesn't work:

>>> class A(int, metaclass=ABCMeta):
@abstractmethod
def frobnicate(self): pass


>>> A()
0

Is this a bug, or an implementation accident, or the expected behavior?

--
https://mail.python.org/mailman/listinfo/python-list

Re: Unexpected Inheritance Problem

[Peter Otten]

On 20/05/2021 06:00, Richard Damon wrote:


class GedcomHead(Gedcom0Tag):
     """GEDCOM 0 HEAD tag"""
     def ___init___(self, *, parent):


An __init__ with three underscores; you must me joking ;)

--
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Annoying message duplication, was Re: Unsubscribe/can't login

[Peter Otten]

On 05/05/2021 20:08, Jan van den Broek wrote:

On 2021-05-05, Peter Otten <__pete...@web.de> wrote:

On 05/05/2021 16:10, Ethan Furman wrote:


I see your messages twice (occasionally with other posters as well).?? I
have no idea how to fix it.?? :(


OK, I'll try another option from Thunderbird's context menu: Followup to
Newsgroup.

Does that appear once or twice?


Here, once.


OK, thanks Jan for bringing this up. From now on I'll try to use the 
"followup" option consistently.


All, if you see duplicate posts more than once in a while you may inform 
me -- private mail is probably preferred by those who don't ;).



--
https://mail.python.org/mailman/listinfo/python-list

[OT] Annoying message duplication, was Re: Unsubscribe/can't login

[Peter Otten]

On 05/05/2021 16:10, Ethan Furman wrote:

I see your messages twice (occasionally with other posters as well).  I 
have no idea how to fix it.  :(


OK, I'll try another option from Thunderbird's context menu: Followup to 
Newsgroup.


Does that appear once or twice?

In theory it should go to the newsgroup only which would mirror it to 
the list.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Unsubscribe/can't login

[Peter Otten]

On 05/05/2021 13:03, Jan van den Broek wrote:

On 2021-05-05, Peter Otten <__pete...@web.de> wrote:

Perhaps there's something wrong on my side, but I'm
seeing this message twice:

Msg-ID: mailman.145.1620211376.3087.python-l...@python.org
Return-Path: __pete...@web.de

and

Msg-ID: mailman.146.1620211381.3087.python-l...@python.org
Return-Path: python-python-l...@m.gmane-mx.org


Sorry for the inconvenience.

Does that happen with all my messages? I'm reading the mailing list via 
news.gmaneio.


When I had a private complaint about duplicate messages some time ago I 
switched from "Reply to All" to "Reply to List" in Thunderbird and was 
hoping that this would avoid the problem.


If it is only this message I may have hit "Reply to All" accidentally.
If it happens with all my messages, maybe there is a Thunderbird/gmane 
user who can tell me how to fix my setup.




--
https://mail.python.org/mailman/listinfo/python-list

Re: Unsubscribe/can't login

[Peter Otten]

On 05/05/2021 05:55, Rasig Kosonmontri wrote:

Hi I wanted to unsubscribe from the python mailing list because I has been
sending me countless mails over the months but when I try to login on my
browser it doesn't recognize this email as a user even though the mails
that are coming it are to this email address
If there's anything thing to fix or anyway to do this manually please let
me know
Thanks


Double-check that you are using the correct email address, then enter it
in the last input field on

https://mail.python.org/mailman/listinfo/python-list

Hit the [Unsubscribe or edit options] button, and on the page that is
now shown hit the [Unsubscribe] button. You should receive an email with
a link that you can open in the browser to complete the process of
unsubscribing.

If that doesn't work try contacting python-list-owner at python.org for
assistance.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Current thinking on required options

[Peter Otten]

On 19/04/2021 11:52, Loris Bennett wrote:

Hi,

I have various small programs which tend to have an interface like the
following example:

   usage: grocli [-h] [-o {check,add,delete}] [-u USERS [USERS ...]] [-g GROUP]

   Command line grouper tool

   optional arguments:
 -h, --helpshow this help message and exit
 -o {check,add,delete}, --operation {check,add,delete}
   operation to apply
 -u USERS [USERS ...], --users USERS [USERS ...]
   users to apply operation to
 -g GROUP, --group GROUP
   group to apply operation to

However, the options -o, -u, and -g are required, not optional.

The documentation

   https://docs.python.org/3/library/argparse.html#required

advises against required options and here

   
https://stackoverflow.com/questions/24180527/argparse-required-arguments-listed-under-optional-arguments

a way of adding a section 'required arguments' to the usage is
described.

I would be interested to know what the general thinking on "required
options" is.  Is there just a better way of designing such interfaces?


For the example above you could realize the operation through subparsers
and switch group and users. In cases where no such "natural" order of
arguments exists I'd have no qualms to use required options. Personally
I've not yet come across such a case.
--
https://mail.python.org/mailman/listinfo/python-list

Re: setting user input equal to target language in Google Translator

[Peter Otten]

On 16/04/2021 19:11, Quentin Bock wrote:

is it possible to set the target language of a translation to be the input
from a user?
I have tried inputting specific abbreviations that would normally be
accepted as the target language but it remains in Icelandic and I would
like to change the target language based on the user's input without
creating hundred of scenarios for each inputted language or country.
Thanks


Hi, Quentin!

When you ask a question it is best to give the code you have. For
beginner problems you may even get a hint from people who haven't used
the library in question. That said, I have just installed googletrans,
and changing the destination language appears to be as easy as

>>> import googletrans as gt
>>> t = gt.Translator()
>>> for language in ["de", "fr", "es"]:
... print(t.translate("Hello, world!", dest=language).text)
...
Hallo Welt!
Bonjour le monde!
¡Hola Mundo!

Does that help?
--
https://mail.python.org/mailman/listinfo/python-list

Re: TIME IN XARRAY

[Peter Otten]

On 15/04/2021 20:20, Chris Angelico wrote:

On Fri, Apr 16, 2021 at 4:16 AM Jorge Conforte  wrote:

I'm using xarray to read netcdf data and I had to time in my data the
values:

xarray.DataArray 'time' (time: 507)>
array(['1979-01-01T00:00:00.0', '1979-02-01T00:00:00.0',
 '1979-03-01T00:00:00.0', ...,
'2021-01-01T00:00:00.0',
 '2021-02-01T00:00:00.0', '2021-03-01T00:00:00.0'],
dtype='datetime64[ns]')


Please, how can I get the years and months values from this array.



Looks like your "time" values are ISO-8601 dates, stored as strings.
You should be able to parse them easily by hand, or hand them to
datetime.datetime.fromisoformat().


The dtype is a strong hint that there is a numpy or pandas datetime
object underneath the misleading repr().

Assuming that x is the column array I'd try

[d.year for d in x]

or

x.map(operator.attrgetter("year"))
--
https://mail.python.org/mailman/listinfo/python-list

Re: Immutable view classes - inherit from dict or from Mapping?

[Peter Otten]

On 12/04/2021 18:01, Andreas R Maier wrote:

Hi,
I have written some classes that represent immutable views on collections (see 
"immutable-views" package on Pypi).

Currently, these view classes inherit from the abstract collection classes such as Mapping, 
Sequence, Set. However, they implement the read-only methods of dict, list and set, which provides 
some more methods compared to the abstract collection classes. For example, class "dict" 
provides copy() or __reversed__() and the newer OR-operator methods all of which are not defined on 
the abstract class "Mapping".

Note that the view classes do not provide any of the modifying methods of "dict" and 
"list", because after all the views are supposed to be immutable.

My question is, should the dict view class inherit from Mapping or from dict, 
and likewise, should the list view class inherit from Sequence or from list?


It's easy to add methods, and hard to remove them -- therefore I'd 
inherit from Mapping rather than dict.


As to the | operator, maybe it could be added to Mapping?


--
https://mail.python.org/mailman/listinfo/python-list

Re: Using pytest, sometimes does not capture stderr

[Peter Otten]

On 05/04/2021 06:25, Cameron Simpson wrote:


If you truly need to test msg() _without_ the file= parameter, you could
monkey patch module_2:

 old_MSG_DESTINATION = module_2.MSG_DESTINATION
 module_2.MSG_DESTINATION = sys.stderr
 # now the module_2 module has an updated reference for sys.stderr
 ...
 msg("a", "message")
 ...
 module_2.MSG_DESTINATION = old_MSG_DESTINATION
 # normality restored


I was about to write "use contextlib.redirect_sterr()", and noted my
error just before hitting send. There is a tool in the stdlib that might
work though:

from unittest import mock

with mock.patch("module_2.MSG_DESTINATION", sys.stderr):
msg("a", "message")
--
https://mail.python.org/mailman/listinfo/python-list

Re: how to separate the strings in a string

[Peter Otten]

On 02/04/2021 12:38, Peter Otten wrote:

On 02/04/2021 11:16, Egon Frerich wrote:

I have a string like

'"ab,c" , def'

and need to separate it into

"ab,c" and "def".

split separates at the first ',':


bl

'"a,bc", def'

bl.split(',')

['"a', 'bc"', ' def']



The initial string looks like it's close enough to the CSV format.
Unfortunately Python's csv module operates on files, not strings -- to
use it you have to wrap the string into a stream:


import csv
import io
next(csv.reader(io.StringIO('"ab,c" , def')))

['ab,c ', ' def']


Apply str.strip() on every part to remove leading and trailing whitespace:



def mysplit(s):

 return [part.strip() for part in next(csv.reader(io.StringIO(s)))]


mysplit('"ab,c" , def')

['ab,c', 'def']



I forgot that the reader() accepts arbitrary iterables. Instead of
io.StringIO(s) you can just write [s].

--
https://mail.python.org/mailman/listinfo/python-list

Re: how to separate the strings in a string

[Peter Otten]

On 02/04/2021 11:16, Egon Frerich wrote:

I have a string like

'"ab,c" , def'

and need to separate it into

"ab,c" and "def".

split separates at the first ',':


bl

'"a,bc", def'

bl.split(',')

['"a', 'bc"', ' def']



The initial string looks like it's close enough to the CSV format.
Unfortunately Python's csv module operates on files, not strings -- to
use it you have to wrap the string into a stream:

>>> import csv
>>> import io
>>> next(csv.reader(io.StringIO('"ab,c" , def')))
['ab,c ', ' def']


Apply str.strip() on every part to remove leading and trailing whitespace:


>>> def mysplit(s):
return [part.strip() for part in next(csv.reader(io.StringIO(s)))]

>>> mysplit('"ab,c" , def')
['ab,c', 'def']

--
https://mail.python.org/mailman/listinfo/python-list

Re: convert script awk in python

[Peter Otten]

On 25/03/2021 08:14, Loris Bennett wrote:


I'm not doing that, but I am trying to replace a longish bash pipeline
with Python code.

Within Emacs, often I use Org mode[1] to generate date via some bash
commands and then visualise the data via Python.  Thus, in a single Org
file I run

   /usr/bin/sacct  -u $user -o jobid -X -S $start -E $end -s COMPLETED -n  | \
   xargs -I {} seff {} | grep 'Efficiency' | sed '$!N;s/\n/ /' | awk '{print $3 " 
" $9}' | sed 's/%//g'

The raw numbers are formatted by Org into a table


   | cpu_eff | mem_eff |
   |-+-|
   |96.6 |   99.11 |
   |   93.43 |   100.0 |
   |91.3 |   100.0 |
   |   88.71 |   100.0 |
   |   89.79 |   100.0 |
   |   84.59 |   100.0 |
   |   83.42 |   100.0 |
   |   86.09 |   100.0 |
   |   92.31 |   100.0 |
   |   90.05 |   100.0 |
   |   81.98 |   100.0 |
   |   90.76 |   100.0 |
   |   75.36 |   64.03 |

I then read this into some Python code in the Org file and do something like

   df = pd.DataFrame(eff_tab[1:], columns=eff_tab[0])
   cpu_data = df.loc[: , "cpu_eff"]
   mem_data = df.loc[: , "mem_eff"]

   ...

   n, bins, patches = axis[0].hist(cpu_data, bins=range(0, 110, 5))
   n, bins, patches = axis[1].hist(mem_data, bins=range(0, 110, 5))

which generates nice histograms.

I decided rewrite the whole thing as a stand-alone Python program so
that I can run it as a cron job.  However, as a novice Python programmer
I am finding translating the bash part slightly clunky.  I am in the
middle of doing this and started with the following:

 sacct = subprocess.Popen(["/usr/bin/sacct",
   "-u", user,
   "-S", period[0], "-E", period[1],
   "-o", "jobid", "-X",
   "-s", "COMPLETED", "-n"],
  stdout=subprocess.PIPE,
 )

 jobids = []

 for line in sacct.stdout:
 jobid = str(line.strip(), 'UTF-8')
 jobids.append(jobid)

 for jobid in jobids:
 seff = subprocess.Popen(["/usr/bin/seff", jobid],
 stdin=sacct.stdout,
 stdout=subprocess.PIPE,
 )


The statement above looks odd. If seff can read the jobids from stdin 
there should be no need to pass them individually, like:


sacct = ...
seff = Popen(
  ["/usr/bin/seff"], stdin=sacct.stdout, stdout=subprocess.PIPE,
  universal_newlines=True
)
for line in seff.communicate()[0].splitlines():
...



 seff_output = []
 for line in seff.stdout:
 seff_output.append(str(line.strip(), "UTF-8"))

 ...

but compared the to the bash pipeline, this all seems a bit laboured.

Does any one have a better approach?

Cheers,

Loris



-Original Message-
From: Cameron Simpson 
Sent: Wednesday, March 24, 2021 6:34 PM
To: Avi Gross 
Cc: python-list@python.org
Subject: Re: convert script awk in python

On 24Mar2021 12:00, Avi Gross  wrote:

But I wonder how much languages like AWK are still used to make new
programs as compared to a time they were really useful.


You mentioned in an adjacent post that you've not used AWK since 2000.
By contrast, I still use it regularly.

It's great for proof of concept at the command line or in small scripts, and
as the innards of quite useful scripts. I've a trite "colsum" script which
does nothing but generate and run a little awk programme to sum a column,
and routinely type "blah  | colsum 2" or the like to get a tally.

I totally agree that once you're processing a lot of data from places or
where a shell script is making long pipelines or many command invocations,
if that's a performance issue it is time to recode.

Cheers,
Cameron Simpson 


Footnotes:
[1]  https://orgmode.org/




--
https://mail.python.org/mailman/listinfo/python-list

Re: How to implement logging for an imported module?

[Peter Otten]

On 15/03/2021 09:47, Robert Latest via Python-list wrote:

Richard Damon wrote:

On 3/8/21 4:16 AM, Robert Latest via Python-list wrote:

Joseph L. Casale wrote:

I couldn't find any information on how to implement logging in a library
that doesn't know the name of the application that uses it. How is that
done?

That's not how it works, it is the opposite. You need to know the name of
its logger, and since you imported it, you do.

[much snipping]


Last word of advice, don't fight it by hacking up or patching (somehow?),
it will simply not work right for any other case even slightly different
than the one you somehow beat into submission.

I didn't waht to hack the logging system, it's just that I wasn't sure of
its design principles. I had hoped that if I set up a logger (including
levels and formatter) in my main app, the loggers in the imported modules
would somwhow automagically follow suit. Now I understand that for each
imported module I must import its logger, too, and decide how to deal with
its messages.



Each instance of the logger inherents from a 'parent' logger, except for the
top level logger which has the name None (as in the singleton of NoneType),
and unless told otherwise will inherit it properties from its parent.

Thus, if you get the root logger with logging.getLogger() you can set
properties there, and unless a child logger has specifical been told not to
inherit or has been specifically given a different value.

General convention is that modules will use their name as the name of their
logger, as that is generally unique.



I must admit I'm still struggling with the very basics of logging. I don't
understand the behavior of the code samples below at all, see comments.
It seems that logging.debug() et al have some side effects that are required
to get a logger to notice its level in the first place.


# Example 1:
# Why does the logger "mylog" require
# a call to the root logger in order to work?


Because logging.debug() implicitly calls basicConfig() if there are no 
handlers yet for the root logger. You should always configure your 
handlers explicitly, usually by a call to logging.basicConfig().




import logging


# - set the root logger's level
# - add a handler to the root logger
logging.basicConfig(level=logging.DEBUG)


mylog = logging.getLogger('foo')


In most cases I want the same log-level for all loggers, so I'd omit the 
line below.



mylog.setLevel(logging.DEBUG)

mylog.debug('1 mylog.debug()')   # prints nothing
logging.debug('2 logging.debug()') # prints nothing
mylog.debug('3 mylog.debug()')   # works


# Example 2:
# Why do I have to call 'logging.debug' before the root
# logger works?

import logging

mylog = logging.getLogger() # now mylog is the root logger
mylog.setLevel(logging.DEBUG) # setting level of root logger

mylog.debug('1 mylog.debug()')   # prints nothing, why?
logging.debug('2 logging.debug()') # works
mylog.debug('3 mylog.debug()')   # works




--
https://mail.python.org/mailman/listinfo/python-list

Re: How to loop over a text file (to remove tags and normalize) using Python

[Peter Otten]

On 10/03/2021 13:19, S Monzur wrote:

I initially scraped the links using beautiful soup, and from those links
downloaded the specific content of the articles I was interested in
(titles, dates, names of contributor, main texts) and stored that
information in a list. I then saved the list to a text file.
https://pastebin.com/8BMi9qjW . I am now trying to remove the html tags
from this text file, and running into issues as mentioned in the previous
post.


As I said in my previous post, when you process the list entries
separately you will probably avoid the problem.

Unfortunately with the format you chose to store your intermediate data
you cannot reconstruct it reliably.

I recommend that you either

(1) avoid the text file and extract the interesting parts from PASoup
directly or

(2) pick a different file format to store the result sets. For
short-term storage pickle
 should work.

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to loop over a text file (to remove tags and normalize) using Python

[Peter Otten]

On 10/03/2021 04:35, S Monzur wrote:

Thanks! I ended up using beautiful soup to remove the html tags and create
three lists (titles of article, publications dates, main body) but am still
facing a problem where the list is not properly storing the main body.
There is something wrong with my code for that section, and any comment
would be really helpful!

  ListFile Text



How did you create that file?

> BeautifulSoup code for removing tags 


print(bodytext[0]) # so here, I'm only getting the first paragraph of the body 
of the first article, not all of the first article

print(bodytext[1]) # here, I'm getting the second paragraph of the first 
article, and not the second article


It may help if you process the individual articles with beautiful soup, 
not the whole list at once.


--
https://mail.python.org/mailman/listinfo/python-list

Re: Button placement

[Peter Otten]

On 09/03/2021 03:08, Bischoop wrote:


I try for a while place the top button in a a middle of column=0 with
span to column2 below so the button being centered to those, 


I'm not sure what you mean, do you perhaps need columnspan=3?


tried with
canvas as well but with no success. Is someone with Tkinter experience
able to advice what I could do?

from tkinter import *
from tkinter import ttk

root = Tk()

root.title('Password Generator')
root.config(padx=10, pady=10)

backg = '#06090f'


# no Canvas
bgs = Button(
root,
text="Password Manager & Generator",
background='#0f2a52', foreground="orange",
font=("Times New Roman", 15, 'bold')
)
bgs.grid(column=0, row=0, pady=10, columnspan=3)


long_t = Label(text='Password Length:')
long_t.grid(column=0, row=1)
choice_n = IntVar()
choice = ttk.Combobox(width=10)
w = [x for x in range(8, 16)]
choice['values'] = w
choice.grid(row=1, column=1)
choice.current()
choice.bind("<>")
password_text = Label(text='Password: ')
password_text.grid(row=2, column=0)
password_entry = Entry(width=20)
password_entry.grid(row=2, column=1)
gen = Button(text='Generate Password')
gen.grid(row=1, column=2, sticky=W)




--
https://mail.python.org/mailman/listinfo/python-list

Re: Choosable dependency

[Peter Otten]

On 06/03/2021 12:43, Manfred Lotz wrote:

On Sat, 6 Mar 2021 12:00:33 +0100
Manfred Lotz  wrote:


Let us say I have a package which reads a TOML file.

I want to give the user of my package the choice to decide if he wants
to use the toml, tomlkit or rtoml package.

So, in case the user chose to use rtoml then there should be an import
only for rtoml, aso.

How could I achieve this?



I got the following minimal example working but presumably my solution
isn't the best way.

xtoml.py:

class Xtoml:
 def __init__(self, which='rtoml'):
 self.which = which

 def parse_toml(self, toml_string):
 if self.which == 'rtoml':
 import rtoml as toml
 elif self.which == 'tomlkit':
 import tomlkit as toml
 else:
 import toml

 return toml.loads(toml_string)


test_xtoml.py:

from xtoml import Xtoml

toml_string = """
[default]

basedir = "/myproject"

"""

def main():
 xtoml = Xtoml('toml')
 parsed_toml = xtoml.parse_toml(toml_string)
 print(parsed_toml)

 xtoml = Xtoml()
 parsed_toml = xtoml.parse_toml(toml_string)
 print(parsed_toml)

if __name__ == "__main__":
 main()


As long as you use a common subset of the functions provided by the 
packages you can just do


def main():
global toml  # if you want to use the toml package elsewhere
 # in your module

chosen_toml = ...  # get package name
toml = importlib.import_module(chosen_toml)

data = toml.loads(toml_string)

However, the user usually doesn't care, so I would recommend that you be 
bold and pick the package you prefer ;)



--
https://mail.python.org/mailman/listinfo/python-list

Re: yield from () Was: Re: weirdness with list()

[Peter Otten]

On 01/03/2021 00:47, Alan Gauld via Python-list wrote:

On 28/02/2021 00:17, Cameron Simpson wrote:


BUT... It also has a __iter__ value, which like any Box iterates over
the subboxes. For MDAT that is implemented like this:

 def __iter__(self):
 yield from ()


Sorry, a bit OT but I'm curious. I haven't seen
this before:

yield from ()

What is it doing?
What do the () represent in this context?


The 0-tuple ;)

yield from x

is syntactic sugar for

for item in x:
yield item

Instead of () you can use any empty iterable.
If x is empty nothing is ever yielded.

--
https://mail.python.org/mailman/listinfo/python-list

Re: weirdness with list()

[Peter Otten]

On 28/02/2021 23:33, Marco Sulla wrote:

On Sun, 28 Feb 2021 at 01:19, Cameron Simpson  wrote:

My object represents an MDAT box in an MP4 file: it is the ludicrously
large data box containing the raw audiovideo data; for a TV episode it
is often about 2GB and a movie is often 4GB to 6GB.
[...]
That length is presented via the object's __len__ method
[...]

I noticed that it was stalling, and investigation revealed it was
stalling at this line:

 subboxes = list(self)

when doing the MDAT box. That box (a) has no subboxes at all and (b) has
a very large __len__ value.

BUT... It also has a __iter__ value, which like any Box iterates over
the subboxes. For MDAT that is implemented like this:

 def __iter__(self):
 yield from ()

What I was expecting was pretty much instant construction of an empty
list. What I was getting was a very time consuming (10 seconds or more)
construction of an empty list.


I can't reproduce, Am I missing something?

marco@buzz:~$ python3
Python 3.6.9 (default, Jan 26 2021, 15:33:00)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

class A:

... def __len__(self):
... return 1024**3
... def __iter__(self):
... yield from ()
...

a = A()
len(a)

1073741824

list(a)

[]




It takes milliseconds to run list(a)


Looks like you need at least Python 3.8 to see this. Quoting
https://docs.python.org/3/whatsnew/3.8.html:

"""
The list constructor does not overallocate the internal item buffer if 
the input iterable has a known length (the input implements __len__). 
This makes the created list 12% smaller on average. (Contributed by 
Raymond Hettinger and Pablo Galindo in bpo-33234.)

"""



--
https://mail.python.org/mailman/listinfo/python-list

Re: weirdness with list()

[Peter Otten]

On 28/02/2021 01:17, Cameron Simpson wrote:

I just ran into a surprising (to me) issue with list() on an iterable
object.

My object represents an MDAT box in an MP4 file: it is the ludicrously
large data box containing the raw audiovideo data; for a TV episode it
is often about 2GB and a movie is often 4GB to 6GB. For obvious reasons,
I do not always want to load that into memory, or even read the data
part at all when scanning an MP4 file, for example to recite its
metadata.

So my parser has a "skip" mode where it seeks straight past the data,
but makes a note of its length in bytes. All good.

That length is presented via the object's __len__ method, because I want
to know that length later and this is a subclass of a suite of things
which return their length in bytes this way.

So, to my problem:

I've got a walk method which traverses the hierarchy of boxes in the MP4
file. Until some minutes ago, it looked like this:

   def walk(self):
 subboxes = list(self)
 yield self, subboxes
 for subbox in subboxes:
   if isinstance(subbox, Box):
 yield from subbox.walk()

somewhat like os.walk does for a file tree.

I noticed that it was stalling, and investigation revealed it was
stalling at this line:

 subboxes = list(self)

when doing the MDAT box. That box (a) has no subboxes at all and (b) has
a very large __len__ value.

BUT... It also has a __iter__ value, which like any Box iterates over
the subboxes. For MDAT that is implemented like this:

 def __iter__(self):
 yield from ()

What I was expecting was pretty much instant construction of an empty
list. What I was getting was a very time consuming (10 seconds or more)
construction of an empty list.

I believe that this is because list() tries to preallocate storage. I
_infer_ from the docs that this is done maybe using
operator.length_hint, which in turn consults "the actual length of the
object" (meaning __len__ for me?), then __length_hint__, then defaults
to 0.

I've changed my walk function like so:

   def walk(self):
 subboxes = []
 for subbox in self:
   subboxes.append(subbox)
 ##subboxes = list(self)


list(iter(self))

should work, too. It may be faster than the explicit loop, but also
defeats the list allocation optimization.


and commented out the former list(self) incantation. This is very fast,
because it makes an empty list and then appends nothing to it. And for
your typical movie file this is fine, because there are never _very_
many immediate subboxes anyway.

But is there a cleaner way to do this?

I'd like to go back to my former list(self) incantation, and modify the
MDAT box class to arrange something efficient. Setting __length_hint__
didn't help: returning NotImplemeneted or 0 had no effect, because
presumably __len__ was consulted first.

Any suggestions? My current approach feels rather hacky.

I'm already leaning towards making __len__ return the number of subboxes
to match the iterator, especially as on reflection not all my subclasses
are consistent about __len__ meaning the length of their binary form;
I'm probably going to have to fix that - some subclasses are actually
namedtuples where __len__ would be the field count. Ugh.

Still, thoughts? I'm interested in any approaches that would have let me
make list() fast while keeping __len__==binary_length.

I'm accepting that __len__ != len(__iter__) is a bad idea now, though.


Indeed. I see how that train wreck happened -- but the weirdness is not
the list behavior.

Maybe you can capture the intended behavior of your class with two
classes, a MyIterable without length that can be converted into MyList
as needed.


--
https://mail.python.org/mailman/listinfo/python-list

Re: comparing two lists

[Peter Otten]

On 25/02/2021 01:42, Davor Levicki wrote:

i have two lists

list1 = ['01:15', 'abc', '01:15', 'def', '01:45', 'ghi' ]
list2 =  ['01:15', 'abc', '01:15', 'uvz', '01:45', 'ghi' ]

and when I loop through the list


list_difference = []
for item in list1:

   if item not in list2:
 list_difference.append(item)


and I managed to get the difference, but I need time as well
because it is a separate item and 'uvz' does not mean to me anything in the 
list with a few thousand entries.
I tried to convert it to the dictionary, but it overwrites with last key:value 
{'01:15' : 'def'}


The problem is underspecified, but here's my guess:

Are the even items unique? If so you can build the dicts with keys and
values swapped:

>>> list1 = ['01:15', 'abc', '01:15', 'def', '01:45', 'ghi' ]
>>> list2 =  ['01:15', 'abc', '01:15', 'uvz', '01:45', 'ghi' ]
>>> d1 = dict(zip(list1[1::2], list1[::2]))
>>> d2 = dict(zip(list2[1::2], list2[::2]))
>>> d1
{'abc': '01:15', 'def': '01:15', 'ghi': '01:45'}
>>> d2
{'abc': '01:15', 'uvz': '01:15', 'ghi': '01:45'}

To calculate the difference:

>>> d1.keys() - d2
{'def'}
>>> {d1[k]: k for k in d1.keys() - d2}
{'01:15': 'def'}
--
https://mail.python.org/mailman/listinfo/python-list

Re: Bug report

[Peter Otten]

On 24/02/2021 22:03, Dan Stromberg wrote:

On Wed, Feb 24, 2021 at 12:58 PM Peter Otten <__pete...@web.de> wrote:


On 24/02/2021 20:36, Carla Molina wrote:
This is not a bug. Have a look at the array's dtype:

  >>> n = 60461826
  >>> a = np.array([1, 50, 100, 150, 200, 250, 300])
  >>> a.dtype
dtype('int32')


I'm getting dtypes of float64.


When you run the snippet above or


import numpy as np

NR = 0.25
N = 60461826

initialINCIDENCE = np.array([1, 50, 100, 150, 200, 250, 300])
initialINCIDENCE = initialINCIDENCE*N/(10*7*NR)


here, i. e. after the division?

initialINCIDENCE*N

should be an int32 array, but dividing by

(10*7*NR)

returns an dtype=float64 array. Switching back to my modified example:

>>> a/42
array([0.02380952, 1.19047619, 2.38095238, 3.57142857, 4.76190476,
   5.95238095, 7.14285714])
>>> _.dtype
dtype('float64')
--
https://mail.python.org/mailman/listinfo/python-list

Re: Bug report

[Peter Otten]

On 24/02/2021 20:36, Carla Molina wrote:

I found the following bug (python 3.9.1) when multiplying an array by
several variables without parentheses; look at the following example:

import numpy as np

NR = 0.25
N = 60461826

initialINCIDENCE = np.array([1, 50, 100, 150, 200, 250, 300])
initialINCIDENCE = initialINCIDENCE*N/(10*7*NR)
print('First result ' +str(initialINCIDENCE))

initialINCIDENCE = np.array([1, 50, 100, 150, 200, 250, 300])
initialINCIDENCE = initialINCIDENCE*(N/(10*7*NR))
print('Second result ' +str(initialINCIDENCE))

The result given is:

First result [   345.49614857  -7267.86283429  10006.94459429   2739.08176
   -4528.78107429 -11796.64390857   5478.16352   ]
Second result [   345.49614857  17274.80742857  34549.61485714
  51824.42228571  69099.22971429  86374.03714286 103648.84457143]

Clearly both are different, and in particular the first one has no sense to
me.


This is not a bug. Have a look at the array's dtype:

>>> n = 60461826
>>> a = np.array([1, 50, 100, 150, 200, 250, 300])
>>> a.dtype
dtype('int32')

A 32-bit integer cannot hold the result of, e. g. 50 * n, the result is
unhelpfully clipped.

One possible fix is to specify the array type:

>>> b = np.array([1, 50, 100, 150, 200, 250, 300], dtype=float)
>>> b * n
array([6.04618260e+07, 3.02309130e+09, 6.04618260e+09, 9.06927390e+09,
   1.20923652e+10, 1.51154565e+10, 1.81385478e+10])
--
https://mail.python.org/mailman/listinfo/python-list

Re: Logging with 2 process in application

[Peter Otten]

On 22/02/2021 06:57, gayatri funde wrote:

On Monday, February 22, 2021 at 11:15:27 AM UTC+5:30, Chris Angelico wrote:

On Mon, Feb 22, 2021 at 4:41 PM gayatri funde  wrote:


On Monday, February 22, 2021 at 10:47:57 AM UTC+5:30, Dan Stromberg wrote:

On Sun, Feb 21, 2021 at 9:10 PM gayatri funde 
wrote:

Hello,

Greetings!! Have a good day!

This is regarding logging issue i am facing in my application. My
requirement is to create log on daily basis while my application is
running, So to do this i am creating new log file at midnight by comparing
the day it starts and enter to new day with the while loop continuously
checking on date.

In my application i have 2 processes, So what my observation is on the day
of start of application both process are able to log into the file, but as
new day starts new file getting created and only main process log are
getting written into new file, and other process is still writing to old
day log.

So could you please help me to understand this issue and it will be great
to know solution if you can help me with this.


Is it possible each process has open a different "file" on the same
pathname, with only one actually being visible in the filesystem?

Linuxes and Unixes allow such things. I am unaware of Windows being
powerful enough to do so, and besides there you'd probably use the Event
Log anyway.

If that's the case, I do not know of a Python logging-module way of
correcting the problem, but you could introduce a third process that is
used just for logging, and have the original two send their log messages to
the new 3rd logging process.



Just to clarify, file names are different as i am appending date to each file 
name. On new day start main process is writing to new day file and another 
process is still writing to old date file. This i am trying on Linux system.

Thanks for suggestion of creating one dedicated process for logging. But if you 
can help to find any is other reason why second process not able to find the 
new day file handler object.
Thank you for your reply.


You'll need to show your code. Most likely, though, BOTH processes
need to independently update to the new file.

ChrisA



I have 2 files mainModule.py and loggingModule.py files as below:
In loggingModule.py file i am managing new log to create , which i am importing 
to mainModule.py file.

loggingModule.py


[...]


def daily_log():
 global firstTime

 try:
 now = datetime.datetime.now()
 log_day = now.day
 initialize_logger("Log_start.log")
 while True:


Unless there is an exception you are forever stuck in this loop...


 currentDate = datetime.datetime.now().day
 time.sleep(60)

 if currentDate != log_day:  # New day started
 initialize_logger("Log_continue.log")

 except Exception as ex:
 exc_type, exc_obj, tb = sys.exc_info()
 template = "An exception of type {0} occurred at {1}. 
Arguments:\n{2!r}"
 message = template.format(type(ex).__name__, tb.tb_lineno, ex.args)
 logging.error(message)


daily_log()


mainModule.py

import Logic_Check
import logging
import multiprocessing
from loggingModule import *


... and as the import implicitly invokes daily_log() you won't get here, 
i. e. the code below is not executed.



def child_Process(var1):
 while True:
 time.sleep(10)
 logging.info('Log from Child Process')

if __name__ == "__main__":
 var1 = "LotsOfSunshine"
 time.sleep(1)
 logging.info("Log from Main process")

 child_Multiprocess = multiprocessing.Process(target=child_Process, 
args=(var1,))
 child_Multiprocess.start()




--
https://mail.python.org/mailman/listinfo/python-list

Re: Is email.message.get() case insensitive for the header name?

[Peter Otten]

On 14/02/2021 21:50, Chris Green wrote:


It isn't clear from the documentation. Does email.message.get() care
about the case of the header it's getting?

I checking mailing list mails and the "List-Id:" header is a bit
'mixed', i.e. it can be List-Id:, or List-ID: or list-id:, will
email.message.get("List-Id:", "unknown") find all of them?

Let's have a look:

>>> import email.message
>>> email.message.get
Traceback (most recent call last):
  File "", line 1, in 
email.message.get
AttributeError: module 'email.message' has no attribute 'get'


OK, you probably meant

>>> email.message.Message.get



Enter the inspect module for a quick glance at the method's source:

>>> import inspect
>>> print(inspect.getsource(email.message.Message.get))
def get(self, name, failobj=None):
"""Get a header value.

Like __getitem__() but return failobj instead of None when the
field
is missing.
"""
name = name.lower()
for k, v in self._headers:
if k.lower() == name:
return self.policy.header_fetch_parse(k, v)
return failobj

Both the `name` argument and the header keys are converted to lowercase
before the comparison, so yes, the method is case-insensitive (whether
it should be casefold() doesn't matter for ascii-strings).
--
https://mail.python.org/mailman/listinfo/python-list

Re: Mutable defaults

[Peter Otten]

On 09/02/2021 16:17, Antoon Pardon wrote:


Most of us know of the perils of mutable default values. So I came up
with the following proof of concept:


[...]

 def copy_defaults(f):

Once you know you need that decorator you won't need it anymore ;)


@copy_defaults
def prepare(value, lst = []):
     for vl in range(value):
     lst.append(vl)
     return lst

print(prepare(2))
print(prepare(3))

Running the above will produce:

[0, 1]
[0, 1, 2]



Because of

@copy_defaults
def prepare(value, lst=[[]]):
for vl in range(value):
lst[0].append(vl)
return lst

you may consider changing it to deepcopy_defaults.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Files can only be modified in IDLE, but not via Powershell

[Peter Otten]

On 08/02/2021 22:33, Stefan Ritter wrote:


Hi,

It would be highly appreciated if you could offer me some advice to
solve a problem I'm currently facing:

I have a Windows 10 ADM64 desktop and a Windows 10 AMD64 Laptop.

I wrote some code to insert text in a .txt-file. It works perfectly on
my laptop.

On my desktop it works only if i run it in IDLE, the text appears
afterwards in the file . However it does not work if run it in
Powershell (or anything else), there are no changes made in the file.

I do not understand why this is the case. Python version, installation
paths and user are the same on desktop and laptop. I searched a bit and
found out that this issue might be caused by my antivirus software
comodo, but even after I replaced it with antivir, an reinstalled python
this didn't change anything.

As mentionned, any help is highly appreciated. If you need any
additional information please let me know!


My crystal ball says Idle and the shell use different working 
directories; you have a relative path, change one file and are looking 
at the other.


If that's not the case -- show us the code.
If there is an error message and a traceback provide that, too.

Use copy-and-paste for both code and error.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Log exception so traceback contains timestamp and level?

[Peter Otten]

On 07/02/2021 16:12, Peter J. Holzer wrote:

On 2021-02-06 21:01:37 -0600, Skip Montanaro wrote:

The logging package can log exceptions and call stacks, but it does
(in my opinion) a suboptimal job of it. Consider this simple example:

import logging
FORMAT = '%(asctime)-15s %(levelname)s %(message)s'
logging.basicConfig(format=FORMAT, force=True)
log.warning("msg", stack_info=True)

2021-02-06 20:46:52,399 WARNING msg
Stack (most recent call last):
   File "", line 1, in 

It formats the warning message just fine, but simply dumps the
traceback onto the stream with no timestamp or level. For my purposes,
this is more important for exceptions. (I'm just using the stack trace
feature as it's easier in a small example.) I would like to call
something like


I suspect that it just adds the stack trace to the message, so that you
are left with a multi-line message.


If you are willing to process all multiline messages in the same way
that simplifies your custom Formatter:

def make_formatter_class(prefix, style):

class PrefixFormatter(logging.Formatter):
prefix_template = logging._STYLES[style][0](prefix)
prefix_uses_time = prefix_template.usesTime()

def usesTime(self):
return self.prefix_uses_time or super().usesTime()

def format(self, record):
# this is supposed to set asctime attribute implicitly:
text = super().format(record)

lines = text.splitlines(True)
if len(lines) > 1:
prefix = self.prefix_template.format(record)
return "".join(
lines[:1] + [prefix + line for line in lines[1:]]
)
return text

return PrefixFormatter


# optional: monkey-patch to affect basicConfig()
logging.Formatter = make_formatter_class(
prefix="X... %(asctime)s %(levelname)s ", style="%"
)
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to accept argparse.log_level parameter and propagate its value to all other modules

[Peter Otten]

On 27/01/2021 20:04, Zoran wrote:

In the same folder I have three files: main.py, app_logger.py and mymodule.py 
with their contents:

# main.py
import argparse
import app_logger
import mymodule
import logging

logger = app_logger.get_logger(__name__)

def log_some_messages():
 logger.debug(f'{__name__} - debug message')
 logger.info(f'{__name__} - info message')
 logger.warning(f'{__name__} - warning message')

if __name__ == '__main__':
 parser = argparse.ArgumentParser(description="Script for executing data quality 
checks.")
 parser.add_argument("--log-level", default='info', choices=['notset', 'debug', 
'info', 'warning', 'error', 'critical'], help="set log level")
 args = parser.parse_args()

 logger.setLevel(eval(f"logging.{args.log_level.upper()}"))

 log_some_messages()
 mymodule.log_some_messages()


#mymodule.py
import logging

from script_logging import app_logger

logger = app_logger.get_logger(__name__)


def log_some_messages():
 logger.debug(f'{__name__} - debug message')
 logger.info(f'{__name__} - info message')
 logger.warning(f'{__name__} - warning message')


#app_logger.py
import logging

_log_format = f"%(asctime)s - [%(levelname)s] - %(name)s - 
(%(filename)s).%(funcName)s(%(lineno)d) - %(message)s"
_log_level = logging.DEBUG

def get_stream_handler():
 stream_handler = logging.StreamHandler()
 stream_handler.setLevel(_log_level)
 stream_handler.setFormatter(logging.Formatter(_log_format))
 return stream_handler

def get_logger(name):
 logger = logging.getLogger(name)
 logger.setLevel(_log_level)
 logger.addHandler(get_stream_handler())
 return logger

When I run it with main.py --log-level debug I get the following output:

2021-01-25 13:21:01,151 - [DEBUG] - __main__ - (main.py).log_some_messages(10) 
- __main__ - debug message
2021-01-25 13:21:01,151 - [INFO] - __main__ - (main.py).log_some_messages(11) - 
__main__ - info message
2021-01-25 13:21:01,152 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:21:01,152 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:21:01,152 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:21:01,152 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

which is OK. If I change --log-level parameter to 'warning', than output is the 
following:

2021-01-25 13:22:12,760 - [WARNING] - __main__ - 
(main.py).log_some_messages(12) - __main__ - warning message
2021-01-25 13:22:12,760 - [DEBUG] - mymodule - 
(mymodule.py).log_some_messages(10) - mymodule - debug message
2021-01-25 13:22:12,761 - [INFO] - mymodule - 
(mymodule.py).log_some_messages(11) - mymodule - info message
2021-01-25 13:22:12,761 - [WARNING] - mymodule - 
(mymodule.py).log_some_messages(12) - mymodule - warning message

As you can see, in main.py log level is set correctly, but in mymodule.py it is 
not.

How to accept argparse.log_level parameter in if __name__ == '__main__': and 
propagate its value to all other modules?


You just have to set the level in the root logger. The easiest way to
achieve that is to forget about your app_logger wrapper around logging
and call

logging.basicConfig(level=yourlevel, format=yourformat)

in  main.py and to create the loggers in main and mymodule

with

logger = logging.getLogger(__name__)


--
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Peter Otten]

On 19/01/2021 10:42, Bischoop wrote:

On 2021-01-19, Peter Otten <__pete...@web.de> wrote:

On 19/01/2021 04:45, Bischoop wrote:


I sat to it again and solved it.


Congratulations!


lil = tuple(set(s)) # list of characters in s

li=[0,0,0,0,0,0]  # list for counted repeats


I see a minor problem here. What happens if s contains more than len(li)
characters?



Thanks, I've created it when came to idea how to solve the problem and
then forgot that part, will have to update it with two additional line
of code.


import timeit


Since you seem interested in performance: imagine that the input is a
very long string. Let's call the length N.

How often do you iterate over its characters?



Does it take time :-) I actually left it because seen guy in thread
were comparing their time results, I'm pretty much aware that mine
solution is time consuming and there are better ways to do it but
I wanted to finish what I started without any additional libraries
like itertools etc in the way that my knowledge allows. The idea with
for this tuple just came up in a first seconds when I look at that
and was not bothering about time or performance especially that later
on as you seen I had different problem to solve and which took me quite
a time and when I look at it today I think how couldn't I came with it
earlier and let myself stuck on it.

I'm happy that despite I've asked here I've finish it practically wthout
additional help and yeah ( Thanks to Stefan who just pointed me to look
at it from different prespective instead pointing what was wrong),
I'll iterate here n = x [x for x in lil], with big string it can make
difference. Now, when you asked me that question I came indeed for better
idea to do this. One loop during which I can append character if not in lil.


How often does Tim's solution?


oh here Im stuck and dont understand what you mean?


[Tim Chase]


  def consecutive_counter(seq):
  # I'm not sure if there's something
  # like this already in itertools
  cur = nada = object()
  count = 0
  for x in seq:
  if x == cur:
  count += 1
  else:
  if cur is not nada:
  yield cur, count
  cur = x
  count = 1
  if cur is not nada:
  yield cur, count

  def longest(seq):
  results = []
  biggest = 0
  for item, count in seq:
  if count > biggest:
  results = [item]
  biggest = count
  elif count == biggest:
  results.append(item)
  return results, biggest



Tim's code iterates over the string once whereas you iterate multiple
times. While this is mainly a question of elegance performance is
affected when you have *nested* loops:


for i in lil:
c = 0
h= lil.index(i)
for letter in s:


If you are unlucky and run into a string with 1000 different characters
the code in your inner loop will execute 1000*1000 times while Tim's
will run 1000 times. This is called O(N*N) and O(N).

Not all loops are obvious,


lil.index(letter)


contains another loop, and you now have O(N*N*N) behavior.

This is the worst case, usually the alphabet (li1) will be smaller, but
avoiding nested loops when possible is still a good habit to get into.

--
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Peter Otten]

On 19/01/2021 04:45, Bischoop wrote:


I sat to it again and solved it.


Congratulations!

> lil = tuple(set(s)) # list of characters in s
>
> li=[0,0,0,0,0,0]  # list for counted repeats

I see a minor problem here. What happens if s contains more than len(li)
characters?


import timeit


Since you seem interested in performance: imagine that the input is a
very long string. Let's call the length N.

How often do you iterate over its characters?
How often does Tim's solution?
--
https://mail.python.org/mailman/listinfo/python-list

Re: count consecutive elements

[Peter Otten]

On 17/01/2021 02:15, Dan Stromberg wrote:

IMO a good set of tests is much more important than type annotations ;)


   def get_longest(string: str) -> typing.Tuple[int, typing.List[str]]:
   """Get the longest run of a single consecutive character."""


May I ask why you artificially limit the signature to str?
If there were a signature, shouldn't it be something like

Iterable[E] --> Tuple[int, List[E]]

where E may be anything that has an equality operation?
As someone who has not yet looked seriously into typing: is there a way
to spell this?


   if not string:
   return (0, [])
   grouped = itertools.groupby(string)
   grouped_with_lengths = [(len(list(value)), key) for key, value in
grouped]
   max_count_and_letter = max(grouped_with_lengths)
   max_count = max_count_and_letter[0]
   result = (max_count, sorted(list_ for count, list_ in
grouped_with_lengths if count == max_count))
   return result


If you want to dabble some more here's a test case you might strive for
your function to pass:

"""
>>> class A:
... def __init__(self, x):
... self.x = x
... def __eq__(self, other):
... return self.x == other.x
... def __repr__(self):
... return self.x

>>> get_longest(map(A, "aaabbcccdaa"))
(3, [a, c])

"""
--
https://mail.python.org/mailman/listinfo/python-list

Re: Class and tkinter problem

[Peter Otten]

On 07/01/2021 08:42, Christian Gollwitzer wrote:

Am 07.01.21 um 08:29 schrieb Paulo da Silva:


Does anybody know why cmd method isn't called when I change the button
state (clicking on it) in this example?
I know that this seems a weird class use. But why doesn't it work?
Thanks.

class C:
 from tkinter import Checkbutton
 import tkinter

 @staticmethod

^^it works if you remove the staticmethod here


 def cmd():
 print("Test")



Maybe there is a bug in tkinter, that it doesn't work with static methods?


It has nothing to do with tkinter, a staticmethod object cannot be
called directly:

>>> class C:
@staticmethod
def cmd(): print("Hello")
cmd()


Traceback (most recent call last):
  File "", line 1, in 
class C:
  File "", line 4, in C
cmd()
TypeError: 'staticmethod' object is not callable

You have to go through the descriptor protocol:

>>> class C:
@staticmethod
def cmd(): print("Hello")


>>> C.cmd()
Hello

I recommend that the OP use a more conventional stye and do the setup
outside the class or, better, in an instance of the class.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter menu item underline syntax [RESOLVED]

[Peter Otten]

On 06/01/2021 22:03, Grant Edwards wrote:


I'm completely baffled by that. Can somebody explain how this
expression is evaluated?

self.callbacks['file->new', underline: 0]

It appears that the dict callbacks is being accessed with the key of
a tuple comprising a string and a slice.

Huh?


You're completely right:

>>> class C:
def __getitem__(self, index): return index


>>> c = C()
>>> underline = 42
>>> c["foo", underline: 0]
('foo', slice(42, 0, None))

The OP's code will most certainly fail at runtime ;)



--
https://mail.python.org/mailman/listinfo/python-list

Re: Tkinter menu item underline syntax [RESOLVED]

[Peter Otten]

On 06/01/2021 22:03, Rich Shepard wrote:

On Thu, 7 Jan 2021, Chris Angelico wrote:


Are you sure that this works? It's syntactically valid, but I don't
think it means what you think it does.


ChrisA,

I'm always open to learning. There's no error generated ... yet the
application doesn' open so it's time to run it through pdb.


Spoiler: unless the name 'underline' is defined you get a NameError:

NameError: name 'underline' is not defined

If it is defined and self.callbacks is a dict you get a TypeError:

TypeError: unhashable type: 'slice'

because that's what a colon means in a context like

x[a:b]

From what you posted I would guess that underline is most likely a 
keyword argument to the add_command() method:


file_menu.add_command(
label='New',
command=self.callbacks['file->new'],
underline=0,
accelerator='Ctrl+N'
)

--
https://mail.python.org/mailman/listinfo/python-list

Re: Funny error message

[Peter Otten]

On 31/12/2020 23:46, Bob van der Poel wrote:


When I run python from the command line and generate an error I get the
following:

Python 3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

z

/home/bob/.local/lib/python3.8/site-packages/requests/__init__.py:89:
RequestsDependencyWarning: urllib3 (1.24.3) or chardet (4.0.0) doesn't
match a supported version!
   warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
Traceback (most recent call last):
   File "", line 1, in 
NameError: name 'z' is not defined

I understand "z in not defined" ... but what's with the warnings?


It looks like Python tries to import requests as a side effect of 
printing the traceback.

Start the interpreter in verbose mode

$ python -v

and trigger the name error again to see which module triggers that import.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Why doesn't collections.Counter support a "key" argument in its constructor?

[Peter Otten]

Saurav Chirania wrote:

> I really like that python's sort method accepts a key function as a
> parameter which can be used to specify how elements should be compared.
> 
> Similarly, we could have a "key" argument which specifies how elements
> should be counted. Let's say we have a list of a million students. I would
> like to do something like:
> c = Counter(students, key = lambda student: student.marks)
> 
> which would return a Counter which maps marks to the number of students
> with that marks. Let's say 29 students in the list had scored 100 marks,
> so one entry in the counter would be key=100, val=29.
> 
> So, why doesn't Counter's constructor support this argument? Are there
> other pythonic ways to do this?

Yes, as the counter won't store the original student there is no advantage 
over the direct translation of your code

Counter(map(lambda: student.marks, students))

or the more elegant variant with a generator expression

Counter(student.marks for student in students)

Functions like max() which do offer a key arg are a bit different as

max(student.marks for student in students)

returns the highest marks whereas

max(students, key=lambda: student.marks)

returns the student with the highest marks. To spell the latter without a 
key arg would require something more complicated like

max((s.marks, i, s) for i, s in enumerate(students))[-1]

If you need to count the marks while remembering the students you would use 
a dict or defaultdict instead of a counter,

d = defaultdict(list)
for s in students:
d[s.marks] = s

or, if the students are sorted by marks, itertools.groupby().

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: PYTHON BUG. deleting elements of list.

[Peter Otten]

Peter Otten wrote:

> If the list is huge you can also delete in reverse order:
> 
> for i in reversed(len(_list)):

Make that reversed(range(len(_list))).

> if discard(_list[i]):
> del _list[i]

Example:

>>> items = ['a', 'b', 'c', 'd', 'e']
>>> for i, item in enumerate(items):
... if item in "bcd":
... del items[i]
... 
>>> items
['a', 'c', 'e']
>>> items = ['a', 'b', 'c', 'd', 'e']
>>> for i in reversed(range(len(items))):
... if items[i] in "bcd":
... del items[i]
... 
>>> items
['a', 'e']


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: PYTHON BUG. deleting elements of list.

[Peter Otten]

Mats Wichmann wrote:

> On 9/7/20 5:01 PM, Driuma Nikita wrote:
> 
> 
>  _list = list(range(50))
>  for i, el in enumerate(_list):
>  del _list[i]
>  print(_list)
> 
> 
> Don't change the the list while you are iterating over it, it messes up
> the iteration. It's not "randomly deleting", it's when next is called to
> fetch the next item, the list has changed.
> 
> One workaround is to iterate over a copy. For example here's using
> slicing to create a new list to iterate over:
> 
> for i, el in enumerate(_list[:]):
>  del _list[i]

As Richard says, this won't work. 
Usually the best approach is to copy the items you want to keep:

_list = [item for item in _list if keep(item)]

If the list is huge you can also delete in reverse order:

for i in reversed(len(_list)):
if discard(_list[i]):
del _list[i]

(If you want to delete /all/ items use _list.clear() or del _list[:])


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: tinker Form question(s)

[Peter Otten]

Steve wrote:

> I am not sure how to post the code for my file here, is copy/paste the
> best way?

Yes. Try to paste only the relevant parts, or, if possible, post a small 
self-contained example that can be run by the reader without further 
editing.

> Is there another? I understand that attachments are stripped
> off.
> 
> """
> The following segment of code returns values of "spec".
> I would like to have it return the associated value of
> "DataLine" with each spec. It would also be nice to understand
> how this code works to hold/return the information.
> 
> In pseudocode, a line something like:
> 
> DataReturned = DataLine + " " + spec)
> New_Specs.append(DataReturned)
> 
> explains what I would like to have returned.

>  return ([spec.get()for spec in New_Specs])

[spec.get() for spec in New_Specs]

is called "list comprehension" and is syntactic sugar for a list-creating 
loop like

result = []
for spec in New_Specs:
result.append(spec.get())


The relevant parts of the loop where you set up the GUI and the New_Specs 
list:

>  New_Specs = []
>  for lineItem in range(len(ThisList)):
>  DataLine, Spec = GetLineByItem(ThisList[y])
...
>  NewSpec = tk.StringVar()
>  New_Specs.append(NewSpec)

First, using range(len(ThisList)) is almost always superfluous as you can 
iterate over the list directly. In your case:

>  New_Specs = []
   for item in ThisList:
>  DataLine, Spec = GetLineByItem(item)
...
>  NewSpec = tk.StringVar()
>  New_Specs.append(NewSpec)

Now, as you need DataLine later-on why not keep a copy? So let's turn 
New_Specs into a list of StringVar, DataLine tuples:

>  New_Specs = []
   for item in ThisList:
>  DataLine, Spec = GetLineByItem(item)
...
>  NewSpec = tk.StringVar()
   New_Specs.append((NewSpec, DataLine))

When the function terminates (there is a mainloop() missing) you can create 
and return the result:

# long form, with a loop
result = []
for spec, dataline in New_Specs:
result.append(dataline + " " + spec.get())
return result

# alternative using a listcomp:
return [dataline + " " + spec.get() for spec, dataline in New_specs]




-- 
https://mail.python.org/mailman/listinfo/python-list

Re: grouping and sorting within groups using another list

[Peter Otten]

Peter Otten wrote:

> group_key = itemgetter(0, 3, 4)
> 
> 
> def sort_key(row, lookup={k: i for i, k in enumerate(sort_list)}):
> return lookup[row[6]]
> 
> 
> result = list(
> chain.from_iterable(
> sorted(group, key=sort_key)
> for _key, group in groupby(rows, key=group_key)
> )
> )

It just occured to me that if the above code is what you want you can omit 
the groupby and sort with a combined key:

sorted(rows, key=lambda row: (group_key(row), sort_key(row)))

(The lambda is for illustration, you could of course merge the two key 
functions into one)

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: grouping and sorting within groups using another list

[Peter Otten]

Larry Martell wrote:

> I have a list of tuples, and I want to group them by 3 items (0, 3, 4)
> and then within each group sort the data by a 4th item (6) using a
> sort order from another list. The list is always ordered by the 3
> grouping items.

>From your description I deduced

from itertools import chain, groupby
from operator import itemgetter
from pprint import pprint

rows = [
('a', 'b', 'c', 'd', 'e', 'f', 'blue', ...),
('a', 'x', 'y', 'd', 'e', 'f', 'green', ...),
('a', 'q', 'w', 'd', 'e', 'f', 'white', ...),
('p', 'x', 'y', 'd', 'e', 'f', 'white', ...),
('p', 'x', 'y', 'd', 'e', 'f', 'blue', ...),
('p', 'x', 'y', 'd', 'e', 'f', 'green', ...),
('z', 'x', 'y', 'm', 'n', 'o', 'green', ...),
('z', 'x', 'y', 'm', 'n', 'o', 'white', ...),
('z', 'x', 'y', 'm', 'n', 'o', 'blue', ...),
]

sort_list = ['blue', 'white', 'green']

wanted = [
('a', 'b', 'c', 'd', 'e', 'f', 'blue', ...),
('a', 'x', 'y', 'd', 'e', 'f', 'white', ...),
('a', 'q', 'w', 'd', 'e', 'f', 'green', ...),
('p', 'x', 'y', 'd', 'e', 'f', 'blue', ...),
('p', 'x', 'y', 'd', 'e', 'f', 'white', ...),
('p', 'x', 'y', 'd', 'e', 'f', 'green', ...),
('z', 'x', 'y', 'm', 'n', 'o', 'blue', ...),
('z', 'x', 'y', 'm', 'n', 'o', 'white', ...),
('z', 'x', 'y', 'm', 'n', 'o', 'green', ...),
]

group_key = itemgetter(0, 3, 4)


def sort_key(row, lookup={k: i for i, k in enumerate(sort_list)}):
return lookup[row[6]]


result = list(
chain.from_iterable(
sorted(group, key=sort_key)
for _key, group in groupby(rows, key=group_key)
)
)
pprint(wanted)
pprint(result)
assert result == wanted


Unfortunately the assertion fails. I'm not sure whether I misunderstood your 
description or if your sample data is incorrect.

> 
> For example, if I have this list:
> rows =
> [('a', 'b', 'c', 'd', 'e', 'f', 'blue', ),
>  ('a', 'x', 'y', 'd', 'e', 'f', 'green', ),
>  ('a', 'q', 'w', 'd', 'e', 'f', 'white', ),
>  ('p', 'x', 'y', 'd', 'e', 'f', 'white', ),
>  ('p', 'x', 'y', 'd', 'e', 'f', ' 'blue', ...),
>  ('p', 'x', 'y', 'd', 'e', 'f', ' 'green', ...),
>  ('z', 'x', 'y', 'm', 'n', 'o', 'green, ...),
>  ('z', 'x', 'y', 'm', 'n', 'o', 'white, ...),
>  ('z', 'x', 'y', 'm', 'n', 'o', 'blue, ...),
> ]
> 
> and I have a list:
> 
> sort_list = ['blue', 'white', 'green']
> 
> Then the result list would be:
> 
> [('a', 'b', 'c', 'd', 'e', 'f', 'blue', ),
>  ('a', 'x', 'y', 'd', 'e', 'f', 'white', ),
>  ('a', 'q', 'w', 'd', 'e', 'f', 'green', ),
>  ('p', 'x', 'y', 'd', 'e', 'f', 'blue', ),
>  ('p', 'x', 'y', 'd', 'e', 'f', ' 'white', ...),
>  ('p', 'x', 'y', 'd', 'e', 'f', ' 'green', ...),
>  ('z', 'x', 'y', 'm', 'n', 'o', 'blue, ...),
>  ('z', 'x', 'y', 'm', 'n', 'o', 'white, ...),
>  ('z', 'x', 'y', 'm', 'n', 'o', 'green, ...),
> ]
> 
> Been trying to do with using groupby but have not been successful.


-- 
https://mail.python.org/mailman/listinfo/python-list