Re: Parallel(?) programming with python

2022-08-08 Thread MRAB

On 2022-08-08 12:20, Stefan Ram wrote:

Andreas Croci  writes:
Basically the question boils down to wether it is possible to have parts 
of a program (could be functions) that keep doing their job while other 
parts do something else on the same data, and what is the best way to do 
this.


   Yes, but this is difficult. If you ask this question here,
   you might not be ready for this.

   I haven't learned it yet myself, but nevertheless tried to
   write a small example program quickly, which might still
   contain errors because of my lack of education.

import threading
import time

def write_to_list( list, lock, event ):
 for i in range( 10 ):
 lock.acquire()
 try:
 list.append( i )
 finally:
 lock.release()
 event.set()
 time.sleep( 3 )

def read_from_list( list, lock, event ):
 while True:
 event.wait()
 print( "Waking up." )
 event.clear()
 if len( list ):
 print( "List contains " + str( list[ 0 ]) + "." )
 lock.acquire()
 try:
 del list[ 0 ]
 finally:
 lock.release()
 else:
 print( "List is empty." )

list = []
lock = threading.Lock()
event = threading.Event()
threading.Thread( target=write_to_list, args=[ list, lock, event ]).start()
threading.Thread( target=read_from_list, args=[ list, lock, event ]).start()

   In basketball, first you must learn to dribble and pass,
   before you can begin to shoot.

   With certain reservations, texts that can be considered
   to learn Python are:

"Object-Oriented Programming in Python Documentation" - a PDF file,
Introduction to Programming Using Python - Y Daniel Liang (2013),
How to Think Like a Computer Scientist - Peter Wentworth (2012-08-12),
The Coder's Apprentice - Pieter Spronck (2016-09-21), and
Python Programming - John Zelle (2009).

When working with threads, you should use queues, not lists, because 
queues do their own locking and can wait for items to arrive, with a 
timeout, if desired:



import queue
import threading
import time

def write_to_item_queue(item_queue):
for i in range(10):
print("Put", i, "in queue.", flush=True)
item_queue.put(i)
time.sleep(3)

# Using None to indicate that there's no more to come.
item_queue.put(None)

def read_from_item_queue(item_queue):
while True:
try:
item = item_queue.get()
except item_queue.Empty:
print("Queue is empty; should've have got here!", flush=True)
else:
print("Queue contains " + str(item) + ".", flush=True)

if item is None:
# Using None to indicate that there's no more to come.
break

item_queue = queue.Queue()

write_thread = threading.Thread(target=write_to_item_queue, 
args=[item_queue])

write_thread.start()

read_thread = threading.Thread(target=read_from_item_queue, 
args=[item_queue])

read_thread.start()

# Wait for the threads to finish.
write_thread.join()
read_thread.join()

print("Finished.")
--
https://mail.python.org/mailman/listinfo/python-list


Re: Exclude 'None' from list comprehension of dicts

2022-08-04 Thread MRAB

On 2022-08-04 12:51, Loris Bennett wrote:

Hi,

I am constructing a list of dictionaries via the following list
comprehension:

   data = [get_job_efficiency_dict(job_id) for job_id in job_ids]

However,

   get_job_efficiency_dict(job_id)

uses 'subprocess.Popen' to run an external program and this can fail.
In this case, the dict should just be omitted from 'data'.

I can have 'get_job_efficiency_dict' return 'None' and then run

   filtered_data = list(filter(None, data))

but is there a more elegant way?


I'm not sure how elegant it is, but:

data = [result for job_id in job_ids if (result := 
get_job_efficiency_dict(job_id)) is not None]

--
https://mail.python.org/mailman/listinfo/python-list


Re: Spotify technology

2022-08-01 Thread MRAB

On 01/08/2022 15:16, Gonzalo V wrote:

Hi everyone!
i need a guide or advice or anything.
I few years ago, i was able to connect to my music receiver via spotify app
from my phone or laptop. You had to connect to receiver and your laptop to
the same wifi network and this is. Years later, that service was deprecated
by spotify conditions. Spotify would only work with some brands, i can
understand that.
Do you know guys, how that technology works?,
it could be replicated via python app?,
Is there some kind a documentation that could help me?.
i cant find anything because i cant find the keyword to start my research
thanks in advance.
Gonzalo form Chile.


Search for: spotify api
--
https://mail.python.org/mailman/listinfo/python-list


Re: Fwd: timedelta object recursion bug

2022-07-28 Thread MRAB

On 28/07/2022 10:54, Ben Hirsig wrote:

Hi, I noticed this when using the requests library in the response.elapsed
object (type timedelta). Tested using the standard datetime library alone
with the example displayed on
https://docs.python.org/3/library/datetime.html#examples-of-usage-timedelta



It appears as though the timedelta object recursively adds its own
attributes (min, max, resolution) as further timedelta objects. I’m not
sure how deep they go, but presumably hitting the recursion limit.




from datetime import timedelta



year = timedelta(days=365)



print(year.max)


   9 days, 23:59:59.99


print(year.max.min.max.resolution.max.min)


   -9 days, 0:00:00



I’m using 3.10.3


It's not recursion, it's a reference cycle. In fact, more than one:

>>> from datetime import timedelta
>>> year = timedelta(days=365)
>>> type(year)

>>> type(year.max)

>>> year.max is year.max.max
True
>>> type(year.min)

>>> year.min is year.min.min
True
--
https://mail.python.org/mailman/listinfo/python-list


Re: random.SystemRandom().randint() inefficient

2022-07-27 Thread MRAB

On 27/07/2022 18:24, Cecil Westerhof via Python-list wrote:

MRAB  writes:


On 27/07/2022 16:43, Cecil Westerhof via Python-list wrote:

"Michael F. Stemper"  writes:


This is orthogonal to your question, but might be of some use to you:

The combination of using len(to_try) as an argument to randint() and
saving the output to a variable named "index" suggests that you might
be setting up to select a random element from to_try, as in:
  something = to_try[index]

If that is the case, you might want to consider using random.choice() instead:

  >>> from random import choice
  >>> to_try = [2,3,5,7,11,13,"seventeen",19]
  >>> choice(to_try)
  2
  >>> choice(to_try)
  'seventeen'
  >>> choice(to_try)
  13
  >>> choice(to_try)
  5
  >>>

Yes, I try to select a random element, but it has also to be removed,
because an element should not be used more as once.
This is the code I use:
 # index = randbelow(len(to_try))
 index = randrange(len(to_try))
 found = permutation[to_try.pop(index)]



When you pop an element from the last, the elements after it need to be
moved down, which takes time.

Try shuffling the list and then popping the now randomly-ordered
elements off the end.


Would shuffling not be a lot more expensive? Especially because I do
not eat the whole list.


You won't know until you time it.
--
https://mail.python.org/mailman/listinfo/python-list


Re: random.SystemRandom().randint() inefficient

2022-07-27 Thread MRAB

On 27/07/2022 16:43, Cecil Westerhof via Python-list wrote:

"Michael F. Stemper"  writes:


This is orthogonal to your question, but might be of some use to you:

The combination of using len(to_try) as an argument to randint() and
saving the output to a variable named "index" suggests that you might
be setting up to select a random element from to_try, as in:
  something = to_try[index]

If that is the case, you might want to consider using random.choice() instead:

  >>> from random import choice
  >>> to_try = [2,3,5,7,11,13,"seventeen",19]
  >>> choice(to_try)
  2
  >>> choice(to_try)
  'seventeen'
  >>> choice(to_try)
  13
  >>> choice(to_try)
  5
  >>>


Yes, I try to select a random element, but it has also to be removed,
because an element should not be used more as once.
This is the code I use:
 # index = randbelow(len(to_try))
 index = randrange(len(to_try))
 found = permutation[to_try.pop(index)]



When you pop an element from the last, the elements after it need to be 
moved down, which takes time.


Try shuffling the list and then popping the now randomly-ordered 
elements off the end.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Object in List : how?

2022-07-23 Thread MRAB

On 23/07/2022 05:28, Khairil Sitanggang wrote:

Hello Expert:

I just started using python. Below is a simple code.  I was trying to check
if, say, NO1 is not in the NODELIST[:].NO
How can I achieve this purpose?

Regards,
-Irfan


class Node:
 def __init__(self):
 self.NO = 0
 self.A = 20

NODE = Node()
NODELIST = []

NODE.NO = 10
NODELIST.append(NODE)

NODE.NO = 20
NODELIST.append(NODE)

NODE.NO = 30
NODELIST.append(NODE)


NO1 = 20
if NO1 not in NODELIST[:].NO  ???


No, you can't do it that way. You have to iterate through the list and 
check each member individually:


if any(NO1 == N.NO for N in NODELIST):

And another thing: you've created only 1 node, and you're changing it 
each time before adding it to the list, so the list ends up with 3 
references to the _same_ object.

--
https://mail.python.org/mailman/listinfo/python-list


Re: spyder v5: invalid file name when selecting interpreter in preferences

2022-07-23 Thread MRAB

On 23/07/2022 18:30, Leif Svalgaard wrote:

error message: invalid file path: C:/Users/leifs/anaconda3/python3105.exe
what is wrong with that?

Is there a file called python3105.exe in the folder 
C:/Users/leifs/anaconda3?


That filename looks wrong to me because the file is usually called 
python.exe.


This page:

https://docs.anaconda.com/anaconda/user-guide/tasks/integration/python-path/

suggests to me that the path is probably more like 
C:\Users\leifs\anaconda3\python.exe (or, if you prefer, 
C:/Users/leifs/anaconda3/python.exe).

--
https://mail.python.org/mailman/listinfo/python-list


Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread MRAB

On 21/07/2022 23:39, Marco Sulla wrote:

I've done this other simple test:

#!/usr/bin/env python3

import tracemalloc
import gc
import pickle

tracemalloc.start()

snapshot1 = tracemalloc.take_snapshot().filter_traces(
    (tracemalloc.Filter(True, __file__), )
)

for i in range(1000):
    pickle.dumps(iter([]))

gc.collect()

snapshot2 = tracemalloc.take_snapshot().filter_traces(
    (tracemalloc.Filter(True, __file__), )
)

top_stats = snapshot2.compare_to(snapshot1, 'lineno')
tracemalloc.stop()

for stat in top_stats:
    print(stat)

The result is:

/home/marco/sources/test.py:14: size=3339 B (+3339 B), count=63 (+63), 
average=53 B
/home/marco/sources/test.py:9: size=464 B (+464 B), count=1 (+1), 
average=464 B
/home/marco/sources/test.py:10: size=456 B (+456 B), count=1 (+1), 
average=456 B
/home/marco/sources/test.py:13: size=28 B (+28 B), count=1 (+1), 
average=28 B


It seems that, after 10 million loops, only 63 have a leak, with only 
~3 KB. It seems to me that we can't call it a leak, no? Probably 
pickle needs a lot more cycles to be sure there's actually a real leakage.
If it was a leak, then the amount of memory used or the counts would 
increase with increasing iterations. If that's not happening, if the 
memory used and the counts stay the roughly the same, then it's probably 
not a leak, unless it's a leak of something that happens only once, such 
as creating a cache or buffer on first use.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Why I fail so bad to check for memory leak with this code?

2022-07-21 Thread MRAB

On 21/07/2022 20:47, Marco Sulla wrote:

I tried to check for memory leaks in a bunch of functions of mine using a
simple decorator. It works, but it fails with this code, returning a random
count_diff at every run. Why?

import tracemalloc
import gc
import functools
from uuid import uuid4
import pickle

def getUuid():
 return str(uuid4())

def trace(func):
 @functools.wraps(func)
 def inner():
 tracemalloc.start()

 snapshot1 = tracemalloc.take_snapshot().filter_traces(
 (tracemalloc.Filter(True, __file__), )
 )

 for i in range(100):
 func()

 gc.collect()

 snapshot2 = tracemalloc.take_snapshot().filter_traces(
 (tracemalloc.Filter(True, __file__), )
 )

 top_stats = snapshot2.compare_to(snapshot1, 'lineno')
 tracemalloc.stop()

 for stat in top_stats:
 if stat.count_diff > 3:
 raise ValueError(f"count_diff: {stat.count_diff}")

 return inner

dict_1 = {getUuid(): i for i in range(1000)}

@trace
def func_76():
 pickle.dumps(iter(dict_1))

func_76()


It's something to do with pickling iterators because it still occurs 
when I reduce func_76 to:


@trace
def func_76():
pickle.dumps(iter([]))
--
https://mail.python.org/mailman/listinfo/python-list


Re: list indices must be integers or slices, not str

2022-07-20 Thread MRAB

On 20/07/2022 12:08, Chris Angelico wrote:

On Wed, 20 Jul 2022 at 20:55, Frank Millman  wrote:


On 2022-07-20 11:37 AM, Chris Angelico wrote:
> On Wed, 20 Jul 2022 at 18:34, Frank Millman  wrote:
>>
>> Hi all
>>
>> C:\Users\E7280>python
>> Python 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64
>> bit (AMD64)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>>   >>>
>>   >>> x = list(range(10))
>>   >>>
>>   >>> '{x[1]}'.format(**vars())
>> '1'
>>   >>>
>>   >>> '{x[-1]}'.format(**vars())
>> Traceback (most recent call last):
>> File "", line 1, in 
>> TypeError: list indices must be integers or slices, not str
>>   >>>
>>
>> Can anyone explain this error? It seems that a negative index is deemed
>> to be a string in this case.
>>
>
> Yeah, that does seem a little odd. What you're seeing is the same as
> this phenomenon:
>
 "{x[1]} {x[spam]}".format(x={1: 42, "spam": "ham"})
> '42 ham'
 "{x[1]} {x[spam]}".format(x={"1": 42, "spam": "ham"})
> Traceback (most recent call last):
>File "", line 1, in 
> KeyError: 1
>
> But I can't find it documented anywhere that digits-only means
> numeric. The best I can find is:
>
> https://docs.python.org/3/library/string.html#formatstrings
> """The arg_name can be followed by any number of index or attribute
> expressions. An expression of the form '.name' selects the named
> attribute using getattr(), while an expression of the form '[index]'
> does an index lookup using __getitem__()."""
>
> and in the corresponding grammar:
>
> field_name::=  arg_name ("." attribute_name | "[" element_index "]")*
> index_string  ::=   +
>
> In other words, any sequence of characters counts as an argument, as
> long as it's not ambiguous. It doesn't seem to say that "all digits is
> interpreted as an integer, everything else is interpreted as a
> string". ISTM that a negative number should be interpreted as an
> integer too, but that might be a backward compatibility break.
>

Thanks for investigating this further. I agree it seems odd.

As quoted above, an expression of the form '[index]' does an index
lookup using __getitem()__.

The only __getitem__() that I can find is in the operator module, and
that handles negative numbers just fine.


In general, __getitem__ is the method used to handle those sorts of lookups:

class X:
 def __getitem__(self, item):
 print("Get item", type(item), item)

"{x[0]} {x[1]} {x[-1]} {x[spam]} {x[1.0]}".format(x=X())

Outside of a format directive, you'd need to quote those:

x[0], x[1], x["spam"]

The distinction is that unquoted bare numbers are interpreted as
integers, not as strings. I'm unable to find the exact definition of
that documented.


Do you think it is worth me raising an issue, if only to find out the
rationale if there is one?



I'd wait for other people's responses first, there may be a better
insight to be found than what I was able to come across.


It's mentioned in PEP 3101 (Advanced String Formatting):

"""
The rules for parsing an item key are very simple. If it starts with a 
digit, then it is treated as a number, otherwise it is used as a string.

"""

I have a vague memory of it being discussed on python-dev, but it was a 
long time ago.

--
https://mail.python.org/mailman/listinfo/python-list


Re: python -W

2022-07-12 Thread MRAB

On 12/07/2022 18:39, אורי wrote:

Is `python -W all` and `python -Wa` the same? And what are the options
for  `python -W`? I didn't find it documented. For example, I'm using
`python -W error::DeprecationWarning` and I didn't see it documented on
https://docs.python.org/3/using/cmdline.html

Is there a way to convert all warnings to exceptions and how?


It's documented on that page.

It says """The action names can be abbreviated as desired and the 
interpreter will resolve them to the appropriate action name. For 
example, -Wi is the same as -Wignore."", so "python -Wa" is the same as 
"python -Walways".


It also says """-Werror# Convert to exceptions""", so "python 
-Werror" (or "python -We") will convert all warnings to exceptions.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Include mailing list

2022-07-09 Thread MRAB

On 09/07/2022 11:08, Mxolisi Mbandezi wrote:

Dear Python developers
Please,include me in your mailing list.
Kind regards
Mxolisi


This tells you how to subscribe to this list:

https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list


Re: NILEARN - WHY THIS CODE THROWS AN ERROR?????

2022-07-08 Thread MRAB

On 08/07/2022 23:20, Avi Gross via Python-list wrote:

Nati Stern has asked several questions here, often about relatively technical 
uses of python code that many of us have never used and still is not providing 
more exact info that tends to be needed before anyone can even think of 
diagnosing the problem.

I have learned to stay away from some such questioners. But I am wondering if 
some people (others too) think this forum is a generalized help desk staffed by 
College Professors with nothing else to do.

Many questions are best handled locally where people can look over your 
shoulder or use the same software and may have some fluency in your native 
language. And sometimes you need to do more investigating on your own, and 
perhaps tell us what you tried and why it was not useful, or we end up making 
endless suggestions and being told we are not working on your real issue and so 
on.

The code below is just babel or maybe babble. Something nested in a loop had a 
problem. Why not try something drastic and look at the  files and PICK ONE and 
use it step by step and see when it fails?

It looks like the code wants to ask for all files then ignore some.

Why you would import numpy repeatedly in a loop is beyond me! LOL!

But which command line failed? My GUESS is:

data = img.get_fdata()


If so, did you try to see the current value of the filename you call "i" in the 
loop and see what name was loaded in what looks like a file ending in .nii in this code:

img = nib.load(path+"/"+i)


You need to proceed step by step and see if any previous steps failed.
But what is possible is you got a file with .nii in middle of the name that 
does not end in .gz, or is not in the format needed.


Indeed, it writes JPEG files whose filename contains the original 
filename (with the ".nii") into the same folder, so if it has already 
been run and produced an output file, the next time it's run, it'll trip 
itself up.


All this would've been clear to the OP if it had printed messages as it 
went.



Good luck,
אבי גרוס

-----Original Message-
From: MRAB 
To: python-list@python.org
Sent: Fri, Jul 8, 2022 4:47 pm
Subject: Re: NILEARN - WHY THIS CODE THROWS AN ERROR?

On 08/07/2022 14:06, נתי שטרן wrote:

fullcode:



import nilearn.plotting as plot
import os,gzip,io
import nibabel as nib
path="C:/users/administrator/desktop/nii"
path2="C:/users/administrator/desktop/nii/out/"
for i in os.listdir(path):
      if(".nii.gz" in i):
          pass
      else:

          if(".nii" in i):
              img = nib.load(path+"/"+i)
              data = img.get_fdata()
              print(data)
              import imageio
              X=0
              for s in data:
                  import numpy
                  aleph=numpy.array(s,dtype=numpy.int8)
                  X=X+1
                  plot.plot_img(aleph)

                  imageio.imwrite("C:\\users\\administrator\\desktop\\nii\\"+i
+str(X)+'.jpg', s)






error:
Data given cannot be loaded because it is not compatible with nibabel
format
<https://netanel.ml>


What's the complete traceback?

It might help you to identify the problem if you add messages to tell
you what it's doing, e.g. what file it's about to load.

Apparently, one of the files is not compatible with nibabel.


--
https://mail.python.org/mailman/listinfo/python-list


Re: NILEARN - WHY THIS CODE THROWS AN ERROR?????

2022-07-08 Thread MRAB

On 08/07/2022 14:06, נתי שטרן wrote:

fullcode:



import nilearn.plotting as plot
import os,gzip,io
import nibabel as nib
path="C:/users/administrator/desktop/nii"
path2="C:/users/administrator/desktop/nii/out/"
for i in os.listdir(path):
 if(".nii.gz" in i):
 pass
 else:

 if(".nii" in i):
 img = nib.load(path+"/"+i)
 data = img.get_fdata()
 print(data)
 import imageio
 X=0
 for s in data:
 import numpy
 aleph=numpy.array(s,dtype=numpy.int8)
 X=X+1
 plot.plot_img(aleph)

 imageio.imwrite("C:\\users\\administrator\\desktop\\nii\\"+i
+str(X)+'.jpg', s)






error:
Data given cannot be loaded because it is not compatible with nibabel
format



What's the complete traceback?

It might help you to identify the problem if you add messages to tell 
you what it's doing, e.g. what file it's about to load.


Apparently, one of the files is not compatible with nibabel.
--
https://mail.python.org/mailman/listinfo/python-list


Re: Subtract n months from datetime [Why?]

2022-06-22 Thread MRAB

On 2022-06-22 20:25, Barry Scott wrote:




On 22 Jun 2022, at 17:59, Paulo da Silva  
wrote:

Às 05:29 de 21/06/22, Paulo da Silva escreveu:

As a general response to some comments ...

Suppose we need to delete records from a database older than ...
Today, it's usual to specify days. For example you have to keep some gov papers 
for 90 days. This seems to come from computers era. In our minds, however, we 
immediately think 90 days=3 months.
For example, one may want to delete some files older than 9 months. It's far 
more intuitive than 270 days.
When we talk about years it is still going. For example I need to keep my 
receipts for 5 years because IRS audits.
Accepting this, it's intuitive, for example, that 3 months before July, 31 is 
April, 30.
The same happens for the years. 5 years before February, 29 is February, 28.


The advantage of 30 days, 90 days etc is that a contract or law does not need 
to tell you
how to deal with the problems of calendar months.

As you say in peoples thoughts that 1 month or 3 months etc. But an accounts 
department
will know how to to the number of days till they have to pay up.

OT, but in the UK, when the Gregorian Calendar was adopted, there were 
complaints.


It's often believed that they were just being superstitious about 
"losing" 11 days, but the truth is that they were complaining that rent 
was paid by the month, but wages by the number of days worked.


That month was a lot shorter, with far fewer working days, yet they were 
still expected to pay the same rent!

--
https://mail.python.org/mailman/listinfo/python-list


Re: Subtract n months from datetime [Why?]

2022-06-22 Thread MRAB

On 2022-06-22 17:59, Paulo da Silva wrote:

Às 05:29 de 21/06/22, Paulo da Silva escreveu:

As a general response to some comments ...

Suppose we need to delete records from a database older than ...
Today, it's usual to specify days. For example you have to keep some gov
papers for 90 days. This seems to come from computers era. In our minds,
however, we immediately think 90 days=3 months.
For example, one may want to delete some files older than 9 months. It's
far more intuitive than 270 days.
When we talk about years it is still going. For example I need to keep
my receipts for 5 years because IRS audits.
Accepting this, it's intuitive, for example, that 3 months before July,
31 is April, 30.
The same happens for the years. 5 years before February, 29 is February, 28.

Again, this is my opinion and that's the way I like it :-)


What makes sense depends on where you're looking from.

It's 28 February, you need to keep it for 5 years, therefore you could 
reason that you can dispose of it on 28 February, 5 years hence.


However, that happens to be a leap year.

Should you still have it on 29 February?
--
https://mail.python.org/mailman/listinfo/python-list


Re: "CPython"

2022-06-21 Thread MRAB

On 2022-06-21 03:38, Paulo da Silva wrote:

Às 02:33 de 21/06/22, Chris Angelico escreveu:

On Tue, 21 Jun 2022 at 11:13, Paulo da Silva
 wrote:


Às 20:01 de 20/06/22, Paulo da Silva escreveu:

Às 18:19 de 20/06/22, Stefan Ram escreveu:

The same personality traits that make people react
to troll postings might make them spread unconfirmed
ideas about the meaning of "C" in "CPython".

The /core/ of CPython is written in C.

CPython is the /canonical/ implementation of Python.

The "C" in "CPython" stands for C.




Not so "unconfirmed"!
Look at this article, I recently read:
https://www.analyticsinsight.net/cpython-to-step-over-javascript-in-developing-web-applications/


There is a sentence in ther that begins with "CPython, short for Core
Python, a reference implementation that other Python distributions are
derived from, ...".

Anyway, I wrote "IMHO".

Do you have any credible reference to your assertion "The "C" in
"CPython" stands for C."?

Thank you.


Well ... I read the responses and they are not touching the point!
I just answered, with my opinion based on articles I have read in the
past. Certainly I could not be sure. That's why I responded as an
opinion (IMHO) and not as an assertion.
Stefan Ram responded with a, at least, not very polite post.
That's why I needed to somehow "defend" why I posted that response, and,
BTW, trying to learn why he said that the C in CPython means "written in C".

I still find very strange, to not say weird, that a compiler or
interpreter has a name based in the language it was written. But, again,
is just my opinion and nothing more.



Not sure why it's strange. The point is to distinguish "CPython" from
"Jython" or "Brython" or "PyPy" or any of the other implementations.

Notice that they are, for example, Jython and not JPython.
There is also Cython that is a different thing.

And YES. You have the right to not feel that as strange.


Jython was originally called JPython.
--
https://mail.python.org/mailman/listinfo/python-list


Re: "CPython"

2022-06-21 Thread MRAB

On 2022-06-21 03:52, Avi Gross via Python-list wrote:


This leads to the extremely important question of what would an implementation 
of Python, written completely in C++, be called?
C++Python
CPython++
C+Python+
DPython
SeaPython?
SeeSeaSiPython


CincPython?

FYI, there's a language called D, so DPython would be written in that.


I don't even want to think fo what sound a C# Python would make.
OK, my apologies to all. Being an interpreted language, it makes sense for a 
good part of the interpreter to include parts made in other languages and also 
add-on libraries in even older languages like FORTRAN.  Quite a few languages, 
including some like R, are also partially based on C in similar ways.

[snip]
--
https://mail.python.org/mailman/listinfo/python-list


Re: "CPython"

2022-06-20 Thread MRAB

On 2022-06-21 02:33, Chris Angelico wrote:

On Tue, 21 Jun 2022 at 11:13, Paulo da Silva
 wrote:


Às 20:01 de 20/06/22, Paulo da Silva escreveu:
> Às 18:19 de 20/06/22, Stefan Ram escreveu:
>>The same personality traits that make people react
>>to troll postings might make them spread unconfirmed
>>ideas about the meaning of "C" in "CPython".
>>
>>The /core/ of CPython is written in C.
>>
>>CPython is the /canonical/ implementation of Python.
>>
>>The "C" in "CPython" stands for C.
>>
>>
>
> Not so "unconfirmed"!
> Look at this article, I recently read:
> 
https://www.analyticsinsight.net/cpython-to-step-over-javascript-in-developing-web-applications/
>
>
> There is a sentence in ther that begins with "CPython, short for Core
> Python, a reference implementation that other Python distributions are
> derived from, ...".
>
> Anyway, I wrote "IMHO".
>
> Do you have any credible reference to your assertion "The "C" in
> "CPython" stands for C."?
>
> Thank you.

Well ... I read the responses and they are not touching the point!
I just answered, with my opinion based on articles I have read in the
past. Certainly I could not be sure. That's why I responded as an
opinion (IMHO) and not as an assertion.
Stefan Ram responded with a, at least, not very polite post.
That's why I needed to somehow "defend" why I posted that response, and,
BTW, trying to learn why he said that the C in CPython means "written in C".

I still find very strange, to not say weird, that a compiler or
interpreter has a name based in the language it was written. But, again,
is just my opinion and nothing more.



Not sure why it's strange. The point is to distinguish "CPython" from
"Jython" or "Brython" or "PyPy" or any of the other implementations.
Yes, CPython has a special place because it's the reference
implementation and the most popular, but the one thing that makes it
distinct from all the others is that it's implemented in C.

And just to make it clear, the interpreter/compiler _itself_ is still 
called "python". "CPython" is a name/term that was applied retroactively 
to that particular implementation when another implementation appeared.



I could, perhaps, create my own interpreter and name it "RosuavPython"
after myself, but when something's made by a team, it's usually more
useful to pick something that is fundamental to it (Brython is
designed to be run in a browser, Jython is written in Python to make
it easy to call on Java classes, etc).

ChrisA


--
https://mail.python.org/mailman/listinfo/python-list


Re: Tkinter - cannot import tklib

2022-06-20 Thread MRAB

On 2022-06-20 23:43, Wolfgang Grafen wrote:

Hello all,

I am an experienced Python user and struggle with following statement:


from tklib import *

Traceback (most recent call last):
File "", line 1, in 
ModuleNotFoundError: No module named 'tklib'

I tried to import tklib as shown above on following of my Python installations: 
Anaconda Python 3.8.3, Portable Python 3.9, Python 3.10.1, Python 2.7.10 with 
the same negative result.

I also tried to find help by googling - without success. Also I did not find a 
python module called tklib to install. Tkinter, ttk is working fine.

There are numerous examples using "from tklib import *" so I assume it works 
for most. In the tk-tutorial below tklib is used without special explanation, so I assume 
it should be installed by default with every python installation which has Tkinter 
integrated.

https://tk-tutorial.readthedocs.io/en/latest/intro/intro.html?highlight=app#

First time that I cannot help myself. Please help, what do I do wrong?

I'm thinking that maybe that tutorial is about writing a module (called 
"tklib") to make it easier to write tkinter applications.


The tkinter "Text" widget, for example, doesn't have its own scrollbars 
(you have to add them separately and link them to the widget), so 
writing a module that defines a text-with-scrollbars widget is a good idea.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Tkinter module test: widget class not inserted in application frame

2022-06-17 Thread MRAB

On 2022-06-17 18:06, Rich Shepard wrote:

On Fri, 17 Jun 2022, MRAB wrote:

You haven't shown the code for common_classes.LabelInput, but I'm guessing 
that the first argument should be the parent.



[snip]
You're passing in the _class_ ConactNameInput, but I'm guessing that it 
should be an _instance_ of that class, in this case, 'self'.


Haven't I done this in the application class?
class NameApplication(tk.Tk):
  def __init__(self, *args, **kwargs):
  super().__init__(*args, **kwargs)
  self.title("Contact Name")
  self.geometry("800x600")
  self.resizable(width = False, height = False)
  ContactNameInput(self).grid(row = 0, column = 0, sticky=('EWNS'))
  self.columnconfigure(0, weight=1)

If not, where do I specify the instance of the ContactNameInput class?


This:

 self.inputs['Last name'] = cc.LabelInput(
 ContactNameInput, 'lname',
 input_class = ttk.Entry,
 input_var = tk.StringVar()
 )

should be this:

 self.inputs['Last name'] = cc.LabelInput(
 self, 'lname',
 input_class = ttk.Entry,
 input_var = tk.StringVar()
 )

Also, this:

 self.inputs['First name'] = cc.LabelInput(
 ContactNameInput, 'fname',
 input_class = ttk.Entry,
 input_var = tk.StringVar()
 )

should be this:

 self.inputs['First name'] = cc.LabelInput(
 self, 'fname',
 input_class = ttk.Entry,
 input_var = tk.StringVar()
 )
--
https://mail.python.org/mailman/listinfo/python-list


Re: Tkinter module test: widget class not inserted in application frame

2022-06-17 Thread MRAB

On 2022-06-17 17:19, Rich Shepard wrote:

I'm not seeing the error source in a small tkinter module I'm testing.

The module code:
---
import tkinter as tk
from tkinter import ttk

import common_classes as cc

class ConactNameInput(tk.Frame):
  def __init__(self, parent, *args, **kwargs):
  super().__init__(parent, *args, **kwargs)
  # A dict to keep track of input widgets
  self.inputs = {}

  self.inputs['Last name'] = cc.LabelInput(
  ContactNameInput, 'lname',
  input_class = ttk.Entry,
  input_var = tk.StringVar()
  )
  self.inputs['Last name'].grid(row = 0, column = 0)
  #
  self.inputs['First name'] = cc.LabelInput(
  ContactNameInput, 'fname',
  input_class = ttk.Entry,
  input_var = tk.StringVar()
  )
  self.inputs['First name'].grid(row = 0, column = 1)

  okay_button = tk.Button(self, text="OK",
  command=self.ok)
  okay_button.pack(side=tk.LEFT, padx=(20, 0), pady=(0, 20))

  cancel_button = tk.Button(self, text="Cancel",
command=self.cancel)
  cancel_button.pack(side=tk.RIGHT, padx=(0, 20), pady=(0, 20))

  def okay_button(self):
  pass

  def cancel_button(self):
  Quitter()

  def get_last_name(self):
  pass

  def get_first_name(self):
  pass


class NameApplication(tk.Tk):
  def __init__(self, *args, **kwargs):
  super().__init__(*args, **kwargs)
  self.title("Contact Name")
  self.geometry("800x600")
  self.resizable(width = False, height = False)
  ContactNameInput(self).grid(row = 0, column = 0, sticky=('EWNS'))
  self.columnconfigure(0, weight=1)

if __name__ == '__main__':
  app = NameApplication()
  app.mainloop()
--

The python error traceback:
--
Traceback (most recent call last):
File "contact_history_name_input.py", line 60, in 
  app = NameApplication()
File "contact_history_name_input.py", line 55, in __init__
  ContactNameInput(self).grid(row = 0, column = 0, sticky=('EWNS'))
File "contact_history_name_input.py", line 17, in __init__
  input_var = tk.StringVar()
File "/home/rshepard/development/BusinessTracker/views/common_classes.py", 
line 46, in __init__
  super().__init__(parent, **kwargs)
File "/usr/lib64/python3.7/tkinter/__init__.py", line 2744, in __init__
  Widget.__init__(self, master, 'frame', cnf, {}, extra)
File "/usr/lib64/python3.7/tkinter/__init__.py", line 2292, in __init__
  BaseWidget._setup(self, master, cnf)
File "/usr/lib64/python3.7/tkinter/__init__.py", line 2262, in _setup
  self.tk = master.tk
AttributeError: type object 'ContactNameInput' has no attribute 'tk'
--

I'm not correctly placing the NameInput class in the NameApplication frame.
What have I missed?

You haven't shown the code for common_classes.LabelInput, but I'm 
guessing that the first argument should be the parent.


You're passing in the _class_ ConactNameInput, but I'm guessing that it 
should be an _instance_ of that class, in this case, 'self'.

--
https://mail.python.org/mailman/listinfo/python-list


Re: fill out bulletins

2022-06-13 Thread MRAB

On 2022-06-13 23:41, jak wrote:
[snip]


If you are interested in seeing what I called "post office bulletin"
(English is not my language and I don't know the name, sorry), you can
find a sample pdf (fillable) but it works badly here:

https://www.guardiacostiera.gov.it/venezia/Documents/Bollettino%20MOD.%20TD123.pdf


Probably best to just call it a "form".
--
https://mail.python.org/mailman/listinfo/python-list


Re: Function to Print a nicely formatted Dictionary or List?

2022-06-09 Thread MRAB

On 2022-06-09 11:43, Dave wrote:

Hi,

Before I write my own I wondering if anyone knows of a function that will print 
a nicely formatted dictionary?

By nicely formatted I mean not all on one line!


It's called "pretty-printing". Have a look at the 'pprint' module.
--
https://mail.python.org/mailman/listinfo/python-list


Re: How to test characters of a string

2022-06-07 Thread MRAB

On 2022-06-07 23:24, Dave wrote:

Yes, it was probably just a typeo on my part.


You've misspelled "typo"!


I’ve now fixed the majority of cases but still got two strings that look 
identical but fail to match, this time (again by 10cc), “I’m Mandy Fly Me”.


Try printing the asciified string:

>>> print(ascii("I’m Mandy Fly Me"))
'I\u2019m Mandy Fly Me'

What you typed above has "smart quotes". Maybe that's also the problem 
in the program: straight single quote/apodtrophe vs smart single quote.



I’m putting money on it being a utf8 problem but I’m stuck on how to handle it. 
It’s probably the single quote in I’m, although it has worked with other songs.

Any ideas?

All the Best
Cheers
Dave

Here is the whole function/method or whatever it’s called in Python:


#
#   checkMusicFiles
#

def checkMusicFiles(theBaseMusicLibraryFolder):
 myArtistDict = []

#
#  Loop thru Artists Folder
#
 myArtistsFoldlerList = getFolderList(theBaseMusicLibraryFolder)
 myArtistCount = 0
 for myArtistFolder in myArtistsFoldlerList:
 print('Artist: ' + myArtistFolder)
#
#  Loop thru Albums Folder
#
 myAlbumList = getFolderList(theBaseMusicLibraryFolder + myArtistFolder)
 for myAlbum in myAlbumList:
 print('Album: ' + myAlbum)

#
#  Loop thru Tracks (Files) Folder
#
 myAlbumPath = theBaseMusicLibraryFolder + myArtistFolder + '/' + 
myAlbum + '/'
 myFilesList = getFileList(myAlbumPath)
 for myFile in myFilesList:
 myFilePath = myAlbumPath + myFile
 myID3 = eyed3.load(myFilePath)
 if myID3 is None:
 continue

 myArtistName = myID3.tag.artist
 if myArtistName is None:
 continue

 myAlbumName = myID3.tag.album
 if myAlbumName is None:
 continue

 myTitleName = myID3.tag.title
 if myTitleName is None:
 continue

 myCompareFileName = myFile[0:-4]
 if myCompareFileName[0].isdigit() and 
myCompareFileName[1].isdigit():
 myCompareFileName = myFile[3:-4]

 if myCompareFileName != myTitleName:
 myLength1 = len(myCompareFileName)
 myLength2 = len(myTitleName)
 print('File Name Mismatch - Artist: [' + myArtistName + '] 
 Album: ['+ myAlbumName + ']  Track: [' + myTitleName + ']  File: [' + 
myCompareFileName + ']')
 if (myLength1 == myLength2):
 print('lengths match: ',myLength1)
 else:
 print('lengths mismatch: ',myLength1,'  ',myLength2)

 print(' ')




 return myArtistsFoldlerList


"myArtistsFoldlerList"?

And so many variables starting with "my". Not wrong; just ... :-)








On 8 Jun 2022, at 00:07, MRAB  wrote:

On 2022-06-07 21:23, Dave wrote:

Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
find.
I have another problem related to this, the following code uses the code you 
just sent. I am getting a files ID3 tags using eyed3, this part seems to work 
and I get expected values in this case myTitleName (Track name) is set to 
“Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” (File 
Name with the Track number prepended). The is digit test works and 
myCompareFileName is set to  “Deadlock Holiday”, so they should match, right?

OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by 10cc)?

[snip]




--
https://mail.python.org/mailman/listinfo/python-list


Re: How to test characters of a string

2022-06-07 Thread MRAB

On 2022-06-07 21:23, Dave wrote:

Thanks a lot for this! isDigit was the method I was looking for and couldn’t 
find.

I have another problem related to this, the following code uses the code you 
just sent. I am getting a files ID3 tags using eyed3, this part seems to work 
and I get expected values in this case myTitleName (Track name) is set to 
“Deadlock Holiday” and myCompareFileName is set to “01 Deadlock Holiday” (File 
Name with the Track number prepended). The is digit test works and 
myCompareFileName is set to  “Deadlock Holiday”, so they should match, right?

OT, but are you sure about that name? Isn't it "Dreadlock Holiday" (by 
10cc)?


[snip]
--
https://mail.python.org/mailman/listinfo/python-list


Re: Help with Python/Eyed3 MusicCDIdFrame method

2022-06-06 Thread MRAB

On 2022-06-06 11:37, Dave wrote:

Hi,

I’m trying to get the ID3 tags of an mp3 file. I trying to use the 
MusicCDIdFrame
  method but I can’t seem to get it right. Here is a code snippet:


  import eyed3
import eyed3.id3
import eyed3.id3.frames
import eyed3.id3.apple
import eyed3.mp3
myID3 = eyed3.load("/Users/Test/Life in the fast lane.mp3")
myTitle = myID3.tag.title
myArtist = myID3.tag.artist
myAlbum = myID3.tag.album
myAlbumArtist = myID3.tag.album_artist
myComposer = myID3.tag.composer
myPublisher = myID3.tag.publisher
myGenre = myID3.tag.genre.name
myCDID = myID3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')

When I run this, I get the following error:

   File "/Documents/Python/Test1/main.py", line 94, in 
 myCDID = myID3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')
AttributeError: 'Mp3AudioFile' object has no attribute 'id3'


Any help or suggestion greatly appreciated.


That line should be:

myCDID = eyed3.id3.frames.MusicCDIdFrame(id=b'MCDI', toc=b'')

Also remember that some attributes might be None, e.g. 'myID3.tag.genre' 
might be None.


Another point: it's probably not worth importing the submodules of 
'eyed3'; you're not gaining anything from it.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Automatic Gain Control in Python?

2022-05-31 Thread MRAB

On 2022-06-01 03:01, Steve GS wrote:
> I started but major issues prevented me this weekend (New Roof, Major 
limb

> broke for pine tree...
> I still have to install pyAudio.

PyPI has wheels only up to Python 3.6, but Christoph Gohlke's site has 
ones for more recent versions:


https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio

> -Original Message-
> From: Python-list  On
> Behalf Of MRAB
> Sent: Tuesday, May 31, 2022 9:47 PM
> To: python-list@python.org
> Subject: Re: Automatic Gain Control in Python?
>
> On 2022-06-01 02:03, Steve GS wrote:
> [snip]
>
> > Maybe you do not understand smart speakers. That is exactly what 
they do.

> > You tell them what podcast/broadcast to play, they get it and play it
> > for you. It is that simple.
> >
> > All I want to do is change the audio levels automatically to make it
> > easier on the ear.
> >
> Did you have a look at the code I posted?
>
> I've found that pyaudio can listen to the Line In input at the same 
time as

> Audacity is recording from it, so you could might be able to use it to
> monitor the level and then tell the smart speaker to turn the volume 
up or

> down.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Automatic Gain Control in Python?

2022-05-31 Thread MRAB

On 2022-06-01 02:03, Steve GS wrote:
[snip]


Maybe you do not understand smart speakers. That is exactly what they do.
You tell them what podcast/broadcast to play, they get it and play it for
you. It is that simple.

All I want to do is change the audio levels automatically to make it easier
on the ear.


Did you have a look at the code I posted?

I've found that pyaudio can listen to the Line In input at the same time 
as Audacity is recording from it, so you could might be able to use it 
to monitor the level and then tell the smart speaker to turn the volume 
up or down.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Tkinter: multicolumn table widget

2022-05-31 Thread MRAB

On 2022-05-31 21:29, Rich Shepard wrote:

On Tue, 31 May 2022, MRAB wrote:


Have a look at the tkinter.ttk.Treeview widget; it can be formatted as a
tree hierarchy, its name suggests, or a multi-column tables, but it
doesn't support multi-line text though, as far as I know.


MRAB,

Thank you, I will.

Each time I add a row to the contacts database table I include a note of
what was discussed and what needs to be done. I'd like to be able to see the
entire note with each contact event.

I'm not committed to using a table so I'm totally open to other approaches.
My needs are few:
- The returned results are read-only.
- The number of rows returned are variable.
- Each row has a contact date, contact type, note, and next contact date.
  The last one isn't necessary to be displayed, but the first three are.
- I want to be able to scroll and view all returned rows.

The note could be displayed partially in the column itself, with the 
full text displayed either in read-only textbox nearby when the row is 
selected (and it's the only selected row), or in the form of a tooltip 
when you hover over it.


There's an example of how to show a tooltip here:

https://stackoverflow.com/questions/3221956/how-do-i-display-tooltips-in-tkinter

As with the tkinter.Text widget, you'll need to add the scrollbar(s) 
separately and then link them.


This I expected.


--
https://mail.python.org/mailman/listinfo/python-list


Re: Tkinter: multicolumn table widget

2022-05-31 Thread MRAB

On 2022-05-31 19:47, Rich Shepard wrote:

My web searches haven't helped me learn how to design a read-only scrollable
table widget displaying rows retrieved from postgres database tables. This
is for my business development application.

I'm writing a view module that displays my contact history with a named
person. The person's last and first name are passed to the database tables
and all the results are displayed in a read-only view.

The view has three vertical sections: person information on top, contact
methods (phones, email) below that, and a scrollable, multicolumn table
at the bottom.

The table columns consist of three tk.StringVal() and one tk.Text() which
usually has multiple rows related to that contact event.

I want to learn how to create the table so each row has the contents of each
of the four columns and I can scroll down and up to look at the history.

All help is certainly appreciated.

Have a look at the tkinter.ttk.Treeview widget; it can be formatted as a 
tree hierarchy, its name suggests, or a multi-column tables, but it 
doesn't support multi-line text though, as far as I know.


As with the tkinter.Text widget, you'll need to add the scrollbar(s) 
separately and then link them.

--
https://mail.python.org/mailman/listinfo/python-list


Re: matplotlib basemap colorbar exception : Given element not contained in the stack

2022-05-29 Thread MRAB

On 2022-05-29 13:57, iMath wrote:

please see the formated code at 
https://stackoverflow.com/questions/72423464/matplotlib-basemap-colorbar-exception-given-element-not-contained-in-the-stack


The problem might be that you're passing "ax=self.ax" when you create 
the basemap, and you have:


self.ax = self.map_canvas.figure.subplots()

According to the docs, ".subplots" doesn't return the axes, it returns a 
_tuple_ of the figure and the axes.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Automatic Gain Control in Python?

2022-05-29 Thread MRAB

On 2022-05-29 16:17, Benjamin Schollnick wrote:

Okay, you are capturing the audio stream as a digital file somewhere, correct?

Why not just right a 3rd party package to normalize the audio levels in the 
digital file?  It’ll be faster, and probably easier than trying to do it in 
real time…

eg. 
https://campus.datacamp.com/courses/spoken-language-processing-in-python/manipulating-audio-files-with-pydub?ex=8
 


Normalizing an audio file with PyDub

Sometimes you'll have audio files where the speech is loud in some portions and 
quiet in others. Having this variance in volume can hinder transcription.

Luckily, PyDub's effects module has a function called normalize() which finds 
the maximum volume of an AudioSegment, then adjusts the rest of the 
AudioSegment to be in proportion. This means the quiet parts will get a volume 
boost.

You can listen to an example of an audio file which starts as loud then goes quiet, 
loud_then_quiet.wav, here 
.

In this exercise, you'll use normalize() to normalize the volume of our file, making 
it sound more like this 
.

or

https://stackoverflow.com/questions/57925304/how-to-normalize-a-raw-audio-file-with-python
 




[snip]

Here's a sample script that uses pyaudio instead of Audacity.

You can check whether the podcast is playing by checking the volume soon 
after it should've started.


Pyaudio can also read and write files.


import pyaudio
import time
import numpy as np

WIDTH = 2
CHANNELS = 2
RATE = 44100
MAX_VOL = 1024

GAIN_STEP = 0.2
LOUDER = 1 + GAIN_STEP
QUIETER = 1 - GAIN_STEP

gain = 1

p = pyaudio.PyAudio()

def callback(data, frame_count, time_info, status):
global gain

# Decode the bytestream
chunk = np.frombuffer(data, dtype=np.int16)

# Adjust the volume.
chunk = (chunk.astype(np.double) * gain).astype(np.int16)

# Adjust the gain according to the current maximum volume.
max_vol = max(chunk)

if max_vol < MAX_VOL:
gain *= LOUDER
elif max_vol > MAX_VOL:
gain *= QUIETER

return (chunk.tobytes(), pyaudio.paContinue)

stream = p.open(format=p.get_format_from_width(WIDTH), channels=CHANNELS,
  rate=RATE, input=True, output=True, stream_callback=callback)

stream.start_stream()

while stream.is_active():
time.sleep(0.1)

stream.stop_stream()
stream.close()

p.terminate()
--
https://mail.python.org/mailman/listinfo/python-list


Re: Automatic Gain Control in Python?

2022-05-28 Thread MRAB

On 2022-05-29 01:17, Steve GS wrote:

"My first thought is you are solving the wrong problem. What seems a better
option would be to get your code to actually connect up to the podcast and
just download the audio directly, rather than trying to get the smart
speaker to play the audio and record it with a microphone."

The smart-speaker is bringing in the podcast by hourly automated commands
and sending by audio cable to a computer which is recording it with
Audacity.  This is an automated system that runs for 48 hours every weekend.
Its output is played live throughout the facility and is also recorded for
replay through the week.

No download to use.

AGC is to happen when the Smart Speaker is playing it, real time.
Any post-record editing would be a horrendous task to say the least.


[snip]
Why would post-record editing be "horrendous"?

Does it record the whole 48 hours into 1 file?

If it's recording each podcast separately, it could process each one 
after recording it, even while the next one is being recorded, and I 
really doubt that processing each one while take long.

--
https://mail.python.org/mailman/listinfo/python-list


Re: starting Python 3.10.4 64-bit

2022-05-27 Thread MRAB

On 2022-05-27 17:18, Rolf Blum wrote:

Hallo,
I installed Python 3.10.4 64-bit successfully. I Also checked in the
Setup to add it to the path.
In the win 8.1 console python 3.7 is started when I enter: python.
This also happened after a reboot and after a repair with the Setup.
How do I start the python console for 3.10.4?
Thanks


You probably also have Python 3.8 on the path, and it's finding that one 
first.


These days it's recommended to use the Python Launcher, "py". That lets 
you specify which version of Python you want to use.

--
https://mail.python.org/mailman/listinfo/python-list


Re: bug in python 3.10.4

2022-05-26 Thread MRAB

On 2022-05-26 02:46, Shuaib Akhtar wrote:

When double clicking a .py file when have python install. It run file but
at a spot of the program it stop running. But using the built-in ide for
python this problem does not happen also any other ide it work fine

When you double-click on a .py file, Windows opens a console window and 
runs the program, and when the program finishes, Windows closes the 
console window. That's normal.

--
https://mail.python.org/mailman/listinfo/python-list


Re: F-string usage in a print()

2022-05-24 Thread MRAB

On 2022-05-24 22:14, Kevin M. Wilson via Python-list wrote:

future_value = 0
for i in range(years):
# for i in range(months):
future_value += monthly_investment
future_value = round(future_value, 2)
# monthly_interest_amount = future_value * monthly_interest_rate
# future_value += monthly_interest_amount
# display the result
print(f"Year = ", years + f"Future value = \n", future_value)When joining a 
string with a number, use an f-string otherwise, code a str() because a implicit convert of an int 
to str causes a TypeError!Well...WTF! Am I not using the f-string function correctly...in the above 
line of code???


There's no implicit conversion. An f-string always gives you a string.

'years' is an int, f"Future value = \n" is a str, and you can't add an 
int and a str.


Maybe you meant this:

print(f"Year = {i}, Future value = {future_value}")

or this:

print(f"Year = {i + 1}, Future value = {future_value}")

These are equivalent to:

print("Year = {}, Future value = {}".format(i, future_value))

and:

print("Year = {}, Future value = {}".format(i + 1, future_value))
--
https://mail.python.org/mailman/listinfo/python-list


Re: oop issue

2022-05-23 Thread MRAB

On 2022-05-23 20:36, Tola Oj wrote:

i am trying to print this code but it keeps giving me this typeerror,
please help. the csv file format i am trying to change into a list is in a
different module.

class invest_crypto:
 crypto_current_rate = 0.05
 client_list = []
 def __init__(self, name, surname, amount_Deposited, amount_to_transfer):
 self.name = name
 self.surname = surname
 self.amount_Deposited = amount_Deposited
 self.amount_to_transfer = amount_to_transfer

 invest_crypto.client_list.append(self)

 def calculate_customer_transfer(self):
 self.customer_transfer = (self.crypto_current_rate * self.
amount_Deposited) + self.amount_Deposited
 return self.customer_transfer

 @classmethod
 def access_client_details(cls):
 with open('C:\\Users\\ojomo\\OneDrive\\Desktop\\myexcel\\
oop_learn.py\\myExperiment.py\\clientDetails.csv', 'r' ) as f:
 reader = csv.DictReader(f)
 clientDetails = list(reader)

 for item in clientDetails:
 invest_crypto(
 name=item.get('name'),
 surname=item.get('surname'),
 amount_Deposited=item.get('amount_deposited'),
 amount_to_transfer=item.get('amount_to_transfer')
 )
 @staticmethod
 def __repr__(self):
 return f"('{self.name}', '{self.surname}', '{self.amount_Deposited}',
'{self.amount_to_transfer}')"


invest_crypto.access_client_details()
print(invest_crypto.client_list())


"this typeerror"? What type error? You haven't shown the traceback.

I'm guessing that it's complaining about the last line, where you're 
calling 'client_list', which is a list. Don't call something that's not 
callable!


The last line should be:

print(invest_crypto.client_list)

(I'm not going to mention the way you're creating instances that append 
themselves onto a list that's on the class; that's just weird, IMHO...)

--
https://mail.python.org/mailman/listinfo/python-list


Re: Now for something completely different...

2022-05-22 Thread MRAB

On 2022-05-23 00:30, Chris Angelico wrote:

On Mon, 23 May 2022 at 09:19, Skip Montanaro  wrote:

That's not too informative (other than its relationship to moi), and I have
room for probably four or five more characters. (I have a graphic artist in
mind, so the space need not strictly be text either.)


Aww, not enough room to say "straight line", because (in Euclidean
space) it's the fastest way from B to A.


Instead of "straight line", "A → B".
--
https://mail.python.org/mailman/listinfo/python-list


Re: Python & nmap

2022-05-19 Thread MRAB

On 2022-05-19 20:28, ^Bart wrote:

You forgot the second line (after 'import nmap' and before 'nm.scan()'):

 nm = nmap.PortScanner()


import nmap
nm = nmap.PortScanner()
nm.scan(hosts='192.168.205.0/24', arguments='-n -sP -PE -PA21,23,80,3389')
hosts_list = [(x, nm[x]['status']['state']) for x in nm.all_hosts()]
for host, status in hosts_list:
   print('{0}:{1}'.host)

And the result is:

Traceback (most recent call last):
File "/home/gabriele/Documenti/Python/nmap.py", line 1, in 
  import nmap
File "/home/gabriele/Documenti/Python/nmap.py", line 2, in 
  nm = nmap.PortScanner()
AttributeError: partially initialized module 'nmap' has no attribute
'PortScanner' (most likely due to a circular import)
  >>>

I'm using the IDLE Shell 3.9.2 on Debian Bullseye+KDE, if I write the
script from command line it works!

When you installed nmap it would've been installed into site-packages, 
but the traceback says "/home/gabriele/Documenti/Python/nmap.py", which 
suggests to me that you called your script "nmap.py", so it's shadowing 
what you installed and is actually trying to import itself!

--
https://mail.python.org/mailman/listinfo/python-list


Re: Issue sending data from C++ to Python

2022-05-18 Thread MRAB

On 2022-05-18 15:08, Pablo Martinez Ulloa wrote:

Hello,

I have been using your C++ Python API, in order to establish a bridge from
C++ to Python. We want to do this, as we have a tactile sensor, which only
has a library developed in C++, but we want to obtain the data in real time
in Python to perform tests with a robotic arm and gripper. The problem we
are facing is with transmitting the data into Python. We are using the
function Py_BuildValue, and it seems to be working, but only for sending
one value at a time, and we want to transmit 54 numbers. When
transmitting a single value retrieved from the sensor into Python in a
double or string format, it works, but when we make the string bigger it
gives us an error with the utf-8 encoding. Also when sending a tuple of
doubles or strings, it works when the tuple is composed of a single
element, but not when we send 2 or more values. Is there any way of fixing
this issue? Or is it just not possible to transmit such a large amount of
data?


I find it easier if I have some code in front of me to 'criticise'!

Have you thought about building and returning, say, a list instead?

/* Error checking omitted. */
PyObject* list;
list = PyList_New(0);
PyList_Append(list, PyLong_FromSsize_t(1));
PyList_Append(list, PyFloat_FromDouble(2.0));
...
--
https://mail.python.org/mailman/listinfo/python-list


Re: Convert the decimal numbers expressed in a `numpy.ndarray` into a matrix representing elements in fractional form

2022-05-17 Thread MRAB

On 2022-05-17 16:21, Peter J. Holzer wrote:

On 2022-05-16 20:48:02 -0400, Dennis Lee Bieber wrote:

On Mon, 16 May 2022 17:22:17 -0700 (PDT), "hongy...@gmail.com"
 declaimed the following:
>a=re.sub(r"'","",a)

Explain what you believe this operation is doing, show us the input and
the output.

The best I can make out of that is that it is looking for single quote
characters within whatever "a" is, and replacing them with nothing.
Something much more understandable, without invoking a regular expression
library (especially when neither the search nor the replacement terms are
regular expressions) with simple string operations...

stripped = "".join(quoted.split("'"))


Whether that's easier to understand it very much in the eye of the
beholder.

As it's just a simple replacement, I would've thought that the 'obvious' 
solution would be:

a = a.replace("'", "")
--
https://mail.python.org/mailman/listinfo/python-list


Re: Non-deterministic set ordering

2022-05-15 Thread MRAB

On 2022-05-16 04:20, Rob Cliffe via Python-list wrote:



On 16/05/2022 04:13, Dan Stromberg wrote:


On Sun, May 15, 2022 at 8:01 PM Rob Cliffe via Python-list 
 wrote:


I was shocked to discover that when repeatedly running the following
program (condensed from a "real" program) under Python 3.8.3

for p in { ('x','y'), ('y','x') }:
 print(p)

the output was sometimes

('y', 'x')
('x', 'y')

and sometimes

('x', 'y')
('y', 'x')

Can anyone explain why running identical code should result in
traversing a set in a different order?


Sets are defined as unordered so that they can be hashed internally to 
give O(1) operations for many tasks.


It wouldn't be unreasonable for sets to use a fixed-by-arbitrary 
ordering for a given group of set operations, but being unpredictable 
deters developers from mistakenly assuming they are ordered.


If you need order, you should use a tuple, list, or something like 
https://grantjenks.com/docs/sortedcontainers/sortedset.html

Thanks, I can work round this behaviour.
But I'm curious: where does the variability come from?  Is it deliberate
(as your answer seems to imply)?  AFAIK the same code within the *same
run* of a program does produce identical results.


Basically, Python uses hash randomisation in order to protect it against 
denial-of-service attacks. (Search for "PYTHONHASHSEED" in the docs.)


It also applied to dicts (the code for sets was based on that for 
dicts), but dicts now remember their insertion order.

--
https://mail.python.org/mailman/listinfo/python-list


Re: "py" command for Linux and Mac?

2022-05-12 Thread MRAB

On 2022-05-12 17:25, Dan Stromberg wrote:

Hi folks.

I heard there's a Windows-like "py" command for Linux (and Mac?).

I'm finally getting to porting a particular project's Python 2.7 code to
3.x, and one of the first steps will probably be changing a lot of "python2
script.py" to use #!/usr/bin/env python2 and chmod +x.  Then we can update
the scripts one at a time to use #!/usr/bin/env python3.

However, would this be Linux-and-Mac-only?  I'm not at all sure this code
will ever move to Windows, but in case it does, would a "py" command work
on all 3 if I use #!/usr/bin/env py?

And if so, where can I find that "py" command for Linux and Mac?

I tried searching for it in Google and on Pypi, but unsurprisingly
searching for "py" gives a buzzillion hits on other things.

It's called the "Python Launcher"; you'll get more helpful results if 
you search for that.

--
https://mail.python.org/mailman/listinfo/python-list


Re: tail

2022-05-08 Thread MRAB

On 2022-05-08 19:15, Barry Scott wrote:




On 7 May 2022, at 22:31, Chris Angelico  wrote:

On Sun, 8 May 2022 at 07:19, Stefan Ram  wrote:


MRAB  writes:

On 2022-05-07 19:47, Stefan Ram wrote:

...

def encoding( name ):
  path = pathlib.Path( name )
  for encoding in( "utf_8", "latin_1", "cp1252" ):
  try:
  with path.open( encoding=encoding, errors="strict" )as file:
  text = file.read()
  return encoding
  except UnicodeDecodeError:
  pass
  return "ascii"
Yes, it's potentially slow and might be wrong.
The result "ascii" might mean it's a binary file.

"latin-1" will decode any sequence of bytes, so it'll never try
"cp1252", nor fall back to "ascii", and falling back to "ascii" is wrong
anyway because the file could contain 0x80..0xFF, which aren't supported
by that encoding.


 Thank you! It's working for my specific application where
 I'm reading from a collection of text files that should be
 encoded in either utf_8, latin_1, or ascii.



In that case, I'd exclude ASCII from the check, and just check UTF-8,
and if that fails, decode as Latin-1. Any ASCII files will decode
correctly as UTF-8, and any file will decode as Latin-1.

I've used this exact fallback system when decoding raw data from
Unicode-naive servers - they accept and share bytes, so it's entirely
possible to have a mix of encodings in a single stream. As long as you
can define the span of a single "unit" (say, a line, or a chunk in
some form), you can read as bytes and do the exact same "decode as
UTF-8 if possible, otherwise decode as Latin-1" dance. Sure, it's not
perfectly ideal, but it's about as good as you'll get with a lot of
US-based servers. (Depending on context, you might use CP-1252 instead
of Latin-1, but you might need errors="replace" there, since
Windows-1252 has some undefined byte values.)


There is a very common error on Windows that files and especially web pages that
claim to be utf-8 are in fact CP-1252.

There is logic in the HTML standards to try utf-8 and if it fails fall back to 
CP-1252.

Its usually the left and "smart" quote chars that cause the issue as they code
as an invalid utf-8.


Is it CP-1252 or ISO-8859-1 (Latin-1)?
--
https://mail.python.org/mailman/listinfo/python-list


Re: tail

2022-05-07 Thread MRAB

On 2022-05-07 19:47, Stefan Ram wrote:

Marco Sulla  writes:

Well, ok, but I need a generic method to get LF and CR for any
encoding an user can input.


   "LF" and "CR" come from US-ASCII. It is theoretically
   possible that there might be some encodings out there
   (not for Unicode) that are not based on US-ASCII and
   have no LF or no CR.


is good for any encoding? Furthermore, is there a way to get the
encoding of an opened file object?


   I have written a function that might be able to detect one
   of few encodings based on a heuristic algorithm.

def encoding( name ):
 path = pathlib.Path( name )
 for encoding in( "utf_8", "latin_1", "cp1252" ):
 try:
 with path.open( encoding=encoding, errors="strict" )as file:
 text = file.read()
 return encoding
 except UnicodeDecodeError:
 pass
 return "ascii"

   Yes, it's potentially slow and might be wrong.
   The result "ascii" might mean it's a binary file.

"latin-1" will decode any sequence of bytes, so it'll never try 
"cp1252", nor fall back to "ascii", and falling back to "ascii" is wrong 
anyway because the file could contain 0x80..0xFF, which aren't supported 
by that encoding.

--
https://mail.python.org/mailman/listinfo/python-list


Re: tail

2022-05-07 Thread MRAB

On 2022-05-07 19:35, Marco Sulla wrote:

On Sat, 7 May 2022 at 19:02, MRAB  wrote:
>
> On 2022-05-07 17:28, Marco Sulla wrote:
> > On Sat, 7 May 2022 at 16:08, Barry  wrote:
> >> You need to handle the file in bin mode and do the handling of line 
endings and encodings yourself. It’s not that hard for the cases you wanted.
> >
> >>>> "\n".encode("utf-16")
> > b'\xff\xfe\n\x00'
> >>>> "".encode("utf-16")
> > b'\xff\xfe'
> >>>> "a\nb".encode("utf-16")
> > b'\xff\xfea\x00\n\x00b\x00'
> >>>> "\n".encode("utf-16").lstrip("".encode("utf-16"))
> > b'\n\x00'
> >
> > Can I use the last trick to get the encoding of a LF or a CR in any 
encoding?
>
> In the case of UTF-16, it's 2 bytes per code unit, but those 2 bytes
> could be little-endian or big-endian.
>
> As you didn't specify which you wanted, it defaulted to little-endian
> and added a BOM (U+FEFF).
>
> If you specify which endianness you want with "utf-16le" or "utf-16be",
> it won't add the BOM:
>
>  >>> # Little-endian.
>  >>> "\n".encode("utf-16le")
> b'\n\x00'
>  >>> # Big-endian.
>  >>> "\n".encode("utf-16be")
> b'\x00\n'

Well, ok, but I need a generic method to get LF and CR for any
encoding an user can input.
Do you think that

"\n".encode(encoding).lstrip("".encode(encoding))

is good for any encoding?
'.lstrip' is the wrong method to use because it treats its argument as a 
set of characters, so it might strip off too many characters. A better 
choice is '.removeprefix'.

Furthermore, is there a way to get the encoding of an opened file object?


How was the file opened?


If it was opened as a text file, use the '.encoding' attribute (which 
just tells you what encoding was specified when it was opened, and you'd 
be assuming that it's the correct one).



If it was opened as a binary file, all you know is that it contains 
bytes, and determining the encoding (assuming that it is a text file) is 
down to heuristics (i.e. guesswork).


--
https://mail.python.org/mailman/listinfo/python-list


Re: tail

2022-05-07 Thread MRAB

On 2022-05-07 17:28, Marco Sulla wrote:

On Sat, 7 May 2022 at 16:08, Barry  wrote:

You need to handle the file in bin mode and do the handling of line endings and 
encodings yourself. It’s not that hard for the cases you wanted.



"\n".encode("utf-16")

b'\xff\xfe\n\x00'

"".encode("utf-16")

b'\xff\xfe'

"a\nb".encode("utf-16")

b'\xff\xfea\x00\n\x00b\x00'

"\n".encode("utf-16").lstrip("".encode("utf-16"))

b'\n\x00'

Can I use the last trick to get the encoding of a LF or a CR in any encoding?


In the case of UTF-16, it's 2 bytes per code unit, but those 2 bytes 
could be little-endian or big-endian.


As you didn't specify which you wanted, it defaulted to little-endian 
and added a BOM (U+FEFF).


If you specify which endianness you want with "utf-16le" or "utf-16be", 
it won't add the BOM:


>>> # Little-endian.
>>> "\n".encode("utf-16le")
b'\n\x00'
>>> # Big-endian.
>>> "\n".encode("utf-16be")
b'\x00\n'
--
https://mail.python.org/mailman/listinfo/python-list


Re: tail

2022-05-06 Thread MRAB

On 2022-05-06 20:21, Marco Sulla wrote:

I have a little problem.

I tried to extend the tail function, so it can read lines from the bottom
of a file object opened in text mode.

The problem is it does not work. It gets a starting position that is lower
than the expected by 3 characters. So the first line is read only for 2
chars, and the last line is missing.

import os

_lf = "\n"
_cr = "\r"
_lf_ord = ord(_lf)

def tail(f, n=10, chunk_size=100):
 n_chunk_size = n * chunk_size
 pos = os.stat(f.fileno()).st_size
 chunk_line_pos = -1
 lines_not_found = n
 binary_mode = "b" in f.mode
 lf = _lf_ord if binary_mode else _lf

 while pos != 0:
 pos -= n_chunk_size

 if pos < 0:
 pos = 0

 f.seek(pos)
 chars = f.read(n_chunk_size)

 for i, char in enumerate(reversed(chars)):
 if char == lf:
 lines_not_found -= 1

 if lines_not_found == 0:
 chunk_line_pos = len(chars) - i - 1
 print(chunk_line_pos, i)
 break

 if lines_not_found == 0:
 break

 line_pos = pos + chunk_line_pos + 1

 f.seek(line_pos)

 res = b"" if binary_mode else ""

 for i in range(n):
 res += f.readline()

 return res

Maybe the problem is 1 char != 1 byte?


Is the file UTF-8? That's a variable-width encoding, so are any of the 
characters > U+007F?


Which OS? On Windows, it's common/normal for UTF-8 files to start with a 
BOM/signature, which is 3 bytes/1 codepoint.

--
https://mail.python.org/mailman/listinfo/python-list


Re: [winreg] How to access "(Default)" key value?

2022-05-03 Thread MRAB

On 2022-05-03 10:31, Nicolas Formichella wrote:

Hello,

I am encountering an issue with winreg, I can't figure out how to read the 
"(Default)" of a registry value

```
def get_chrome_version() -> str:
 with winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER) as registry:
 with winreg.OpenKeyEx(registry, 
r"Software\Microsoft\Windows\CurrentVersion\App Paths\chrome.exe",
   access=winreg.KEY_READ) as key:
 value, _ = winreg.QueryValueEx(key, "(default)")
 return value
```

Taking the example above it raises a FileNotFoundError :

```
Traceback (most recent call last):
   File "C:/Users/noulo/dev/py/chromedriver_dl/main.py", line 14, in 
get_chrome_version
 value, _ = winreg.QueryValueEx(key, "(default)")
FileNotFoundError: [WinError 2] The system cannot find the file specified
```

(Trying "Path" instead of "(Default)" works)

Although the value definitely exists as shown by Powershell

```
Get-Item -Path  "HKCU:/Software/Microsoft/Windows/CurrentVersion/App 
Paths/chrome.exe" | Select-Object -ExpandProperty Property
```

Returns :

```
(default)
Path
```

Is this a bug I should report, or if not, what is the way, using winreg, to access the 
"(Default)" value of a key?


Use 'QueryValue' with an empty string:

value = winreg.QueryValue(key, "")

or None:

value = winreg.QueryValue(key, None)
--
https://mail.python.org/mailman/listinfo/python-list


Re: Fwd: Python Question re Executing a Script (dn)

2022-05-02 Thread MRAB

On 2022-05-02 17:23, Dennis Lee Bieber wrote:

On Mon, 2 May 2022 20:33:02 +1200, dn 
declaimed the following:


Perhaps an MS-Win user can help the OP, please?


Not really -- at least from my viewpoint there is not enough
information to perform any diagnoses... Other than to recommend that, if
the OP initially posted to the list/newsgroup, they should ensure any
replies also go to the same (I don't respond to what should be group
replies that appear in my personal mail -- which is what the OP did for me
also).

{Hmmm, appears the quoted content is below your signature block, and my
client doesn't quote standard signatures and below... time for some nasty
cut}


From: Brent Hunter 
Date: Mon, 2 May 2022 00:30:22 +




I was recently running a Windows 10 machine and I believe it was running Python 3.8.  All I did was create a batch file 
titled Start-AIG.bat which simply contained the following: "pythonw AIG.py".  It started a python program 
titled "AIG.py" and the Python dialog box was displayed on my screen, running all day and night.  I set up 
Windows to run this batch file upon startup and it worked fine.  I remember having to do a bunch of research before I 
learned that I needed to put "pythonw AIG.py" in the batch file as opposed to "python AIG.py".




[snip]
FTR, I've already posted the suggestion to try the Python Launcher 
"pyw", and the OP replied off-list that it worked.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Difference in Setup Between Windows 10 Running Python 3.9 and Windows 11 Running Python 3.10

2022-05-01 Thread MRAB

On 2022-05-02 02:56, Brent Hunter wrote:

Hello,

I was recently running a Windows 10 machine Python 3.9.  I simply created a batch file titled "Start-AIG.bat" which 
simply contained the following: "pythonw AIG.py".  It started a python program titled "AIG.py" and the Python 
dialog box was displayed on my screen, running all day and night.  I set up Windows to run this batch file upon startup and it 
worked fine.  I remember having to do a bunch of research before I learned that I needed to put "pythonw AIG.py" in the 
batch file as opposed to "python AIG.py".

However, my old windows 10 desktop crashed and I have a new Windows 11 machine. 
 I installed the latest stable version of Python, which is 3.10.  Now, when I 
try to execute the same batch file, it doesn't launch the app.  It could be 
because I'm using Windows 11 or could be because I now have a newer version of 
Python.

Does anyone have any ideas what I should do to get the Python script running on 
my new machine?

The problem is probably that the Python folder isn't in Windows' search 
path, but the recommended thing to do nowadays on Windows is to use the 
Python Launcher "py.exe" (or "pyw.exe" for no console window) instead:


pyw AIG.py
--
https://mail.python.org/mailman/listinfo/python-list


Re: Style for docstring

2022-04-22 Thread MRAB

On 2022-04-23 00:25, Rob Cliffe via Python-list wrote:

I don't use docstrings much; instead I put a line or two of comments
after the `def ` line.
But my practice in such situations is as per the OP's 3rd suggestion, e.g.
      # Returns True if .
I'm curious as to why so many people prefer "Return" to "Returns".
Checking out help() on a few functions in the stdlib, they all used
"Return" or a grammatical equivalent, so this does seem to be a Python
cultural thing.  But why?  To me, "Returns" begins a description as to
what the function does, whereas "Return" is an imperative.  But who is
it addresed to?  Is a function considered to be a sentient entity that
can respond to a command?  Is it an invocation to the lines of code
following the docstring: "Do this!" Might not the programmer mistakenly
think (if only for a moment) that the imperative is addressed to him?


Maybe it's because the function name is often also an imperative, e.g.:

>>> import re
>>> help(re.search)
Help on function search in module re:

search(pattern, string, flags=0)
Scan through string looking for a match to the pattern, returning
a Match object, or None if no match was found.


Note "Scan", not "scans".


I was going to use 'print' as the example:

>>> help(print)
Help on built-in function print in module builtins:

print(...)
print(value, ..., sep=' ', end='\n', file=sys.stdout, flush=False)

Prints the values to a stream, or to sys.stdout by default.
Optional keyword arguments:
file:  a file-like object (stream); defaults to the current sys.stdout.
sep:   string inserted between values, default a space.
end:   string appended after the last value, default a newline.
flush: whether to forcibly flush the stream.


but it says "Prints", not "Print"...
--
https://mail.python.org/mailman/listinfo/python-list


Re: Having trouble getting Hello World to appear

2022-04-21 Thread MRAB

On 2022-04-22 02:57, Greg wrote:

I downloaded and installed the auto version of the software.


"auto version"?


I go to the director C:\google-python-exercises> *python hello.py*

I am running Windows.



What am I doing incorrectly?


I don't know, because you didn't say what did or didn't happen.

Did it say it couldn't find Python?

If yes, try the Python Launcher instead:

py hello.py




I had the zip file installed under my One Drive and then moved it to my C
drive


--
https://mail.python.org/mailman/listinfo/python-list


Re: Why does datetime.timedelta only have the attributes 'days' and 'seconds'?

2022-04-19 Thread MRAB

On 2022-04-19 19:23, Dennis Lee Bieber wrote:

On Tue, 19 Apr 2022 15:51:09 +0200, "Loris Bennett"
 declaimed the following:


If I am merely trying to represent part a very large number of seconds
as a number of years, 365 days per year does not seem that controversial


The Explanatory Supplement to the Astronomical Almanac (table 15.3)
defines the /day/ as 24hrs->1440mins->86400secs   BUT defines the 
Julian
/year/ as 365.25 days.


So, a /day/ is a solar day, as distinct from a sidereal day.

What's the difference?

Well, a "sidereal day" is how long it takes for the Earth to rotate on 
its axis, but as it's also orbiting the Sun in the same direction, 
midday won't happen until a little later, hence "solar day".


[snip]
--
https://mail.python.org/mailman/listinfo/python-list


Re: Python app on a Mac

2022-04-15 Thread MRAB

On 2022-04-15 11:51, Alan Gauld wrote:

I've just migrated from a Linux PC to a Mac mini running Monterey.

I have a Python GUI(Tkinter) app that I wrote on Linux
and have managed to get working on MacOS except

When I start the program I get a Terminal window as well
as the GUI. On Windows I'd run it with pythonw and in Linux
I just created a desktop launcher to avoid that. But there
is no pythonw on MacOS nor a simple way I can find to
launch apps from the desktop.

Does anyone know how to launch a Python program from the
desktop without a Terminal window (or at least with an
iconified one!) Does it require Applescript or somesuch?


There's an answer here:

https://stackoverflow.com/questions/50792048/running-a-python-script-without-opening-terminal
--
https://mail.python.org/mailman/listinfo/python-list


Re: Why does datetime.timedelta only have the attributes 'days' and 'seconds'?

2022-04-14 Thread MRAB

On 2022-04-14 16:22, Jon Ribbens via Python-list wrote:

On 2022-04-14, Paul Bryan  wrote:

I think because minutes and hours can easily be composed by multiplying
seconds. days is separate because you cannot compose days from seconds;
leap seconds are applied to days at various times, due to
irregularities in the Earth's rotation.


That's an argument that timedelta should *not* have a 'days' attribute,
because a day is not a fixed number of seconds long (to know how long
a day is, you have to know which day you're talking about, and where).
It's an undocumented feature of timedelta that by 'day' it means '86400
seconds'.


When you're working only with dates, timedelta not having a 'days' 
attribute would be annoying, especially when you consider that a day is 
usually 24 hours, but sometimes 23 or 25 hours (DST).

--
https://mail.python.org/mailman/listinfo/python-list


Re: Error at https://docs.python.org/3/search.html?q=f+string_keywords=yes=default

2022-04-13 Thread MRAB

On 2022-04-14 02:07, Chris Angelico wrote:

On Thu, 14 Apr 2022 at 11:00, Kevin M. Wilson via Python-list
 wrote:






MS Edge settings are displayed in the first picture, the error I encountered is 
the second picture...not sure how I get around this!I reloaded the browser 
after checking the settings for JavaScript...confused.

Kevin



No attachments here. What's the actual issue?


I've tried it myself:

1. Go to https://docs.python.org/3/

2. Type something into the Quick search box

3. Click Go

I think the problem is that the panel on the left is empty, and it's not 
obvious how to navigate out of the search results page. (The answer is 
to click on the "Documentation" part of "Documentation » Search".)

--
https://mail.python.org/mailman/listinfo/python-list


Re: the downloaded version is not being detected

2022-04-08 Thread MRAB

On 2022-04-08 17:28, Putsala Bhavani Maha Laxmi wrote:

The purpose of this mail is the problem encountered while using it through an 
IDE platform. While using an IDE this version of python is not getting detected 
even though it is already downloaded and a suggestion box to install python 
appears .

I kindly request you to look into this issue.


"An IDE platform"? Which IDE platform? Which OS?
--
https://mail.python.org/mailman/listinfo/python-list


Re: Issues

2022-04-08 Thread MRAB

On 2022-04-08 20:35, Stevenson, John B via Python-list wrote:

Hello,

As a quick disclaimer, I am sorry if you have received this message multiple 
times over from me. I've been having technical difficulties trying to reach 
this email. Thank you.

I'm trying to install Python on a computer so that I can use it for various tasks for my 
job, like mapping and programming. But it's not downloading the necessary files into the 
right repository for me to run Python commands in a command prompt. I can open the Python 
app just fine, but I cannot use it in the terminal, and this messes with pip and prevents 
me from doing my task. What can I do to fix this? Error sent back is "'python' is 
not recognized as an internal or external command, operable program or batch file." 
Thank you.


Try the Python Launcher instead by typing "py" instead of "python".
--
https://mail.python.org/mailman/listinfo/python-list


Re: Comparing sequences with range objects

2022-04-07 Thread MRAB

On 2022-04-07 16:16, Antoon Pardon wrote:

Op 7/04/2022 om 16:08 schreef Joel Goldstick:

On Thu, Apr 7, 2022 at 7:19 AM Antoon Pardon  wrote:

I am working with a list of data from which I have to weed out duplicates.
At the moment I keep for each entry a container with the other entries
that are still possible duplicates.

The problem is sometimes that is all the rest. I thought to use a range
object for these cases. Unfortunatly I sometimes want to sort things
and a range object is not comparable with a list or a tuple.

So I have a list of items where each item is itself a list or range object.
I of course could sort this by using list as a key function but that
would defeat the purpose of using range objects for these cases.

So what would be a relatively easy way to get the same result without wasting
too much memory on entries that haven't any weeding done on them.

--
Antoon Pardon.
--
https://mail.python.org/mailman/listinfo/python-list

I'm not sure I understand what you are trying to do, but if your data
has no order, you can use set to remove the duplicates


Sorry I wasn't clear. The data contains information about persons. But not
all records need to be complete. So a person can occur multiple times in
the list, while the records are all different because they are missing
different bits.

So all records with the same firstname can be duplicates. But if I have
a record in which the firstname is missing, it can at that point be
a duplicate of all other records.


This is how I'd approach it:

# Make a list of groups, where each group is a list of potential duplicates.
# Initially, all of the records are potential duplicates of each other.
records = [list_of_records]

# Split the groups into subgroups according to the first name.
new_records = []

for group in records:
subgroups = defaultdict(list)

for record in group:
subgroups[record['first_name']].append(record)

# Records without a first name could belong to any of the subgroups.
missing = subgroups.pop(None, [])

for record in missing:
for subgroup in subgroups.values():
subgroup.extend(missing)

new_records.extend(subgroups.values())

records = new_records

# Now repeat for the last name, etc.
--
https://mail.python.org/mailman/listinfo/python-list


Re: The Cython compiler is 20 years old today !

2022-04-05 Thread MRAB

On 2022-04-05 04:47, Dan Stromberg wrote:

On Mon, Apr 4, 2022 at 7:42 AM Stefan Behnel  wrote:


Dear Python community,

it's now 20 years since Greg Ewing posted his first announcement of Pyrex,
the tool that is now known and used under the name Cython.

https://mail.python.org/pipermail/python-list/2002-April/126661.html



That's a cool anniversary.

I often shake my head in wonder when I see people still writing extension
modules in C instead of Cython.


Some of us do it because we enjoy it, much like doing crosswords or 
sudoku or wordle, but with an end result! :-)

--
https://mail.python.org/mailman/listinfo/python-list


Re: pyinstaller is not a internal or external command

2022-04-04 Thread MRAB

On 2022-04-04 20:28, Andrew Pierson wrote:

I ran pip install pyinstaller fine but after then I type in pyinstaller
and it says pyinstaller is not a internal or external command


Have you tried using the Python launcher?
py pyinstaller ...
--
https://mail.python.org/mailman/listinfo/python-list


Re: code blocks in Python

2022-03-25 Thread MRAB

On 2022-03-25 10:12, jlfivn wrote:

On Monday, 24 November 2003 at 23:54:49 UTC, John Roth wrote:

"Hung Jung Lu"  wrote in message
news:8ef9bea6.03112...@posting.google.com...
> Skip Montanaro  wrote in message
news:...
> >
> I come back again to repeat it one more time: the compile() function
> already exists and works in Python. Before you do your posting, please
> think about the compile() function first. (Do I need to say it one
> more time? I don't mind.)
So what?
Functionality is not added to Python simply because it looks like
a logical extension of something else that already exists. First,
you need to show a compelling use case.
So far, I've seen one thing in your proposal: dynamic binding
of free variables in a function, rather than static binding. All
questions of syntax aside, please show me why this matters,
bearing in mind that I've never programmed in a language
that has this, and am not going to be convinced by references
to such languages.
While I'm not *the* person that has to be convinced (that's Guido),
I'm probably representative. If you don't manage a compelling
case for why dynamic binding is a useful option, then you're not going
to get anywhere with this proposal.
By the way - if I understand the guts of the proposal, the compile
function has nothing to do with it, and wouldn't be used to implement
it in any case.
John Roth


One use case is modifying the underlying bytecode of the code block to allow 
features that are not implemented or should not be implemented in python but 
still make sense in context. For example python-goto is a module that allows 
you to add goto statements to python code. Currently it has to be applied as a 
decorator to a function which then needs to be called. Being able to create a 
code block and then run it instead of having to rap it in a function could be 
very useful is such cases.
I propose the following syntax:

code_block :  #start a code block
 print("foobar")

#the code block would be executed immediately
#would print "foobar"

code_block as my_code: #save the code to the named variable;does not execute 
immediately
 print("foobar")
#no output but code is in my_code

exec(my_code)
#would print "foobar"

@with_goto  #the 
code block would be past to the decorator then executed
code_block:
 print("ignore the fact that I'm using goto")
 goto .cheat
 print("goto is a bad idea")
 label .cheat

#output :"ignore the fact that I'm using goto"

That must be some kind of record, replying to a post from over 18 
_years_ ago! :-)

--
https://mail.python.org/mailman/listinfo/python-list


Re: Problem slicing a list with the C API

2022-03-12 Thread MRAB

On 2022-03-12 21:24, Jen Kris via Python-list wrote:

I have a C API project where I have to slice a list into two parts.   
Unfortunately the documentation on the slice objects is not clear enough for me 
to understand how to do this, and I haven’t found enough useful info through 
research.  The list contains tuple records where each tuple consists of a 
dictionary object and a string.

The relevant part of the Python code is:

half_slice = int(len(dictdata) * 0.5)
subdata_a = dictdata[half_slice:]
subdata_b = dictdata[:half_slice]

This is what I’ve done so far with the C API:

int64_t Calc_Slices(PyObject* pDictdata, int64_t records_count)
{
long round_half = records_count * 0.5;
PyObject* half_slice = PyLong_FromLong(round_half);

PyObject* slice = PySlice_New(PyLong_FromLong(0), half_slice, 0);
PyObject* subdata_a = PyObject_GetItem(pDictddata, slice);

return 0;
}

On the final line (subdata_a) I get a segfault.  I know that the second 
parameter of  PyObject_GetItem is a “key” and I suspect that’s where the 
problem comes from, but I don’t understand what a key is in this context.

The code shown above omits error handling but none of the objects leading up to 
the final line is null, they all succeed.

Thanks for any ideas.


Use PySequence_GetSlice to slice the list.

Also, why use floats when you can use integers?

long round_half = records_count / 2;

(In Python that would be half_slice = len(dictdata) // 2.)
--
https://mail.python.org/mailman/listinfo/python-list


Re: always return the same pdf

2022-03-07 Thread MRAB

On 2022-03-07 14:08, Gonzalo V wrote:

Hello everyone.
i had upload a Django app to an ubuntu 18.04 server and it gives me the
same pdf everytime the view is called. To generate the pdf it receipts
differents string buy it gives me the same pdf. Could you give some idea
what is happening?

thanks everyone
@never_cached
def generar_pdf(request):
 prueba = request.session.get('contenedor')
 cantidad_preguntas=prueba['cantidad_preguntas']
 archivo_salida = open("prueba.tex","w")

archivo_salida.write("\\documentclass[10pt,oneside,letterpaper]{article}")
 archivo_salida.write("\\usepackage[utf8x]{inputenc}")

   ##hace mas y mas cosas sin importancia con latex que funcionan bien

 archivo_a_descargar = open("prueba.pdf","rb") #
 respuesta =
HttpResponse(archivo_a_descargar,content_type='application/pdf')
 respuesta['Content-Disposition'] = 'attachment; filename="{0}"'.format(
archivo_a_descargar.name)

 return respuesta
Saludos,
Gonzalo


You're using relative paths. Are you sure that they are pointing to the 
correct files?


Is it actually generating the PDF?

You might think that when it generates the PDF it overwrites any 
existing file of that name but is it? Is it simply giving you the PDF 
that's already there?

--
https://mail.python.org/mailman/listinfo/python-list


Re: C API PyObject_CallFunctionObjArgs returns incorrect result

2022-03-07 Thread MRAB

On 2022-03-07 17:05, Jen Kris wrote:

Thank you MRAB for your reply.

Regarding your first question, pSentence is a list.  In the nltk 
library, nltk.word_tokenize takes a string, so we convert sentence to 
string before we call nltk.word_tokenize:


>>> sentence = " ".join(sentence)
>>> pt = nltk.word_tokenize(sentence)
>>> print(sentence)
[ Emma by Jane Austen 1816 ]

But with the C API it looks like this:

PyObject *pSentence = PySequence_GetItem(pSents, sent_count);
PyObject* str_sentence = PyObject_Str(pSentence); // Convert to string

; See what str_sentence looks like:
PyObject* repr_str = PyObject_Repr(str_sentence);
PyObject* str_str = PyUnicode_AsEncodedString(repr_str, "utf-8", "~E~");
const char *bytes_str = PyBytes_AS_STRING(str_str);
printf("REPR_String: %s\n", bytes_str);

REPR_String: "['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']"

So the two string representations are not the same – or at least the   
PyUnicode_AsEncodedString is not the same, as each item is surrounded 
by single quotes.


Assuming that the conversion to bytes object for the REPR is an 
accurate representation of str_sentence, it looks like I need to strip 
the quotes from str_sentence before “PyObject* pWTok = 
PyObject_CallFunctionObjArgs(pNltk_WTok, str_sentence, 0).”


So my questions now are (1) is there a C API function that will 
convert a list to a string exactly the same way as ‘’.join, and if not 
then (2) how can I strip characters from a string object in the C API?



Your Python code is joining the list with a space as the separator.

The equivalent using the C API is:

    PyObject* separator;
    PyObject* joined;

    separator = PyUnicode_FromString(" ");
    joined = PyUnicode_Join(separator, pSentence);
    Py_DECREF(sep);



Mar 6, 2022, 17:42 by pyt...@mrabarnett.plus.com:

On 2022-03-07 00:32, Jen Kris via Python-list wrote:

I am using the C API in Python 3.8 with the nltk library, and
I have a problem with the return from a library call
implemented with PyObject_CallFunctionObjArgs.

This is the relevant Python code:

import nltk
from nltk.corpus import gutenberg
fileids = gutenberg.fileids()
sentences = gutenberg.sents(fileids[0])
sentence = sentences[0]
sentence = " ".join(sentence)
pt = nltk.word_tokenize(sentence)

I run this at the Python command prompt to show how it works:

sentence = " ".join(sentence)
pt = nltk.word_tokenize(sentence)
print(pt)

['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']

type(pt)



This is the relevant part of the C API code:

PyObject* str_sentence = PyObject_Str(pSentence);
// nltk.word_tokenize(sentence)
PyObject* pNltk_WTok = PyObject_GetAttrString(pModule_mstr,
"word_tokenize");
PyObject* pWTok = PyObject_CallFunctionObjArgs(pNltk_WTok,
str_sentence, 0);

(where pModule_mstr is the nltk library).

That should produce a list with a length of 7 that looks like
it does on the command line version shown above:

['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']

But instead the C API produces a list with a length of 24, and
the REPR looks like this:

'[\'[\', "\'", \'[\', "\'", \',\', "\'Emma", "\'", \',\',
"\'by", "\'", \',\', "\'Jane", "\'", \',\', "\'Austen", "\'",
\',\', "\'1816", "\'", \',\', "\'", \']\', "\'", \']\']'

I also tried this with PyObject_CallMethodObjArgs and
PyObject_Call without success.

Thanks for any help on this.

What is pSentence? Is it what you think it is?
To me it looks like it's either the list:

['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']

or that list as a string:

"['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']"

and that what you're tokenising.
-- 
https://mail.python.org/mailman/listinfo/python-list




--
https://mail.python.org/mailman/listinfo/python-list


Re: C API PyObject_CallFunctionObjArgs returns incorrect result

2022-03-06 Thread MRAB

On 2022-03-07 00:32, Jen Kris via Python-list wrote:

I am using the C API in Python 3.8 with the nltk library, and I have a problem 
with the return from a library call implemented with 
PyObject_CallFunctionObjArgs.

This is the relevant Python code:

import nltk
from nltk.corpus import gutenberg
fileids = gutenberg.fileids()
sentences = gutenberg.sents(fileids[0])
sentence = sentences[0]
sentence = " ".join(sentence)
pt = nltk.word_tokenize(sentence)

I run this at the Python command prompt to show how it works:

sentence = " ".join(sentence)
pt = nltk.word_tokenize(sentence)
print(pt)

['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']

type(pt)



This is the relevant part of the C API code:

PyObject* str_sentence = PyObject_Str(pSentence);
// nltk.word_tokenize(sentence)
PyObject* pNltk_WTok = PyObject_GetAttrString(pModule_mstr, "word_tokenize");
PyObject* pWTok = PyObject_CallFunctionObjArgs(pNltk_WTok, str_sentence, 0);

(where pModule_mstr is the nltk library).

That should produce a list with a length of 7 that looks like it does on the 
command line version shown above:

['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']

But instead the C API produces a list with a length of 24, and the REPR looks 
like this:

'[\'[\', "\'", \'[\', "\'", \',\', "\'Emma", "\'", \',\', "\'by", "\'", \',\', "\'Jane", "\'", \',\', "\'Austen", "\'", 
\',\', "\'1816", "\'", \',\', "\'", \']\', "\'", \']\']'

I also tried this with PyObject_CallMethodObjArgs and PyObject_Call without 
success.

Thanks for any help on this.


What is pSentence? Is it what you think it is?
To me it looks like it's either the list:

['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']

or that list as a string:

"['[', 'Emma', 'by', 'Jane', 'Austen', '1816', ']']"

and that what you're tokenising.
--
https://mail.python.org/mailman/listinfo/python-list


Re: Help: Unable to find IDLE folder inside the Python Lib folder

2022-03-05 Thread MRAB

On 2022-03-05 20:36, Deji Olofinboba via Python-list wrote:


Dear Python officer,
Please I am new to programming. I have justinstalled the python 3.10.2. After 
the installation, I was able to locate thePython Shell but unable to locate 
IDLE despite checking it before downloading in the python installation folder. 
I also reinstalled Python and checked IDLE; still the IDLE is still missing. 
Please. explain to me how I can retrieve the python IDLE.
Thank You,


If you're on Windows 10, type "idle" into the search box on the taskbar.
Alternatively, look in python_folder\Lib\idlelib for idle.bat if you're 
on an older version of Windows.

--
https://mail.python.org/mailman/listinfo/python-list


Re: [python-uk] Fwd: It Will Never Work in Theory: live!

2022-03-04 Thread MRAB

On 2022-03-04 20:48, BELAHCENE Abdelkader wrote:

hi,
I receive a lot of email from the python-list, I want to disable it, when I
want to read the email, I want to go to he List.
Please How to disable  it.
Regards


This page explains how to unsubscribe:

https://mail.python.org/mailman/listinfo/python-list


Le jeu. 3 mars 2022 à 18:05, Steve Holden  a écrit :


A communication from my good friend Greg Wilsin (instigator of the
Software Carpentry workshops) which may be of use to some. It certainly
looks like great value for money.

Kind regards,
Steve


-- Forwarded message -
From: Greg Wilson 
Date: Tue, Mar 1, 2022 at 5:11 PM
Subject: It Will Never Work in Theory: live!
To: Steve Holden 


Hi Steve,

On April 27, It Will Never Work in Theory is running two sets of online
lightning talks from leading software engineering researchers in which
they’ll summarize actionable findings from their work for practitioners.
Tickets are now on sale at https://neverworkintheory.org/, and all money
raised will to go Books for Africa. I hope to see you there, and if you
could help spread the word or help sponsor it by matching money raised
from ticket sales, we'd be very grateful.

Cheers,

Greg

___
python-uk mailing list
python...@python.org
https://mail.python.org/mailman/listinfo/python-uk



--
https://mail.python.org/mailman/listinfo/python-list


Re: One-liner to merge lists?

2022-02-27 Thread MRAB

On 2022-02-27 08:51, Barry Scott wrote:




On 22 Feb 2022, at 09:30, Chris Angelico  wrote:

On Tue, 22 Feb 2022 at 20:24, Frank Millman mailto:fr...@chagford.com>> wrote:


Hi all

I think this should be a simple one-liner, but I cannot figure it out.

I have a dictionary with a number of keys, where each value is a single
list -


d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}


I want to combine all values into a single list -


ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']


I can do this -


a = []
for v in d.values():

...   a.extend(v)
...

a

['aaa', 'bbb', 'ccc', 'fff', 'ggg']

I can also do this -


from itertools import chain
a = list(chain(*d.values()))
a

['aaa', 'bbb', 'ccc', 'fff', 'ggg']




Is there a simpler way?



itertools.chain is a good option, as it scales well to arbitrary
numbers of lists (and you're guaranteed to iterate over them all just
once as you construct the list). But if you know that the lists aren't
too large or too numerous, here's another method that works:


sum(d.values(), [])

['aaa', 'bbb', 'ccc', 'fff', 'ggg']

It's simply adding all the lists together, though you have to tell it
that you don't want a numeric summation.


If you code is performance sensitive do not use sum() as it creates lots of tmp 
list that are deleted.

I have an outstanding ticket at work to replace all use of sum() on lists as 
when we profiled it
stands out as a slow operation. We have a lots of list of list that we need to 
flatten.


I think that 'sum' uses '__add__' but not '__iadd__'.

If it copied the given value, if present, and then used '__iadd__', if 
present, wouldn't that speed it up?

--
https://mail.python.org/mailman/listinfo/python-list


Re: C API PyObject_Call segfaults with string

2022-02-10 Thread MRAB

On 2022-02-10 20:00, Jen Kris via Python-list wrote:

With the help of PyErr_Print() I have it solved.  Here is the final code (the 
part relevant to sents):

    Py_ssize_t listIndex = 0;
    pListItem = PyList_GetItem(pFileIds, listIndex);
    pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
    pListStr = PyBytes_AS_STRING(pListStrE); // Borrowed pointer

    // Then:  sentences = gutenberg.sents(fileid) - this is a sequence item
    PyObject *c_args = Py_BuildValue("s", pListStr);
    PyObject *args_tuple = PyTuple_New(1);
    PyTuple_SetItem(args_tuple, 0, c_args);

    pSents = PyObject_CallObject(pSentMod, args_tuple);

    if ( pSents == 0x0){
    PyErr_Print();
    return return_value; }

As you mentioned yesterday, CallObject needs a tuple, so that was the problem.  
Now it works.

You also asked why I don't just use pListStrE.  I tried that and got a long 
error message from PyErr_Print.  I'm not far enough along in my C_API work to 
understand why, but it doesn't work.

Thanks very much for your help on this.


You're encoding a Unicode string to a UTF-8 bytestring:

pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");

then pointing to the bytes of that UTF-8 bytestring:

pListStr = PyBytes_AS_STRING(pListStrE); // Borrowed pointer

then making a Unicode string from those UTF-8 bytes:

PyObject *c_args = Py_BuildValue("s", pListStr);

You might was well just use the original Unicode string!

Try this instead:

   Py_ssize_t listIndex = 0;
   pListItem = PyList_GetItem(pFileIds, listIndex);
   //> pListItem?

   pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem, 0);
   //> pSents+?

   if (pSents == 0x0){
   PyErr_Print();
   return return_value;
   }




Feb 9, 2022, 17:40 by songofaca...@gmail.com:


On Thu, Feb 10, 2022 at 10:37 AM Jen Kris  wrote:



I'm using Python 3.8 so I tried your second choice:

pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);

but pSents is 0x0.  pSentMod and pListItem are valid pointers.



It means exception happened.
If you are writing Python/C function, return NULL (e.g. `if (pSents ==
NULL) return NULL`)
Then Python show the exception and traceback for you.


--
https://mail.python.org/mailman/listinfo/python-list


Re: GCP Copy Files - Data Export

2022-02-10 Thread MRAB

On 2022-02-10 17:20, BmoreIT wrote:

I did a data export from Google to export all company data - Google Data Export

It shows the root folder and to download, I run this command (it automatically 
enters this command)

gsutil -m cp -r \ "gs://takeout-export-myUniqueID" \.
But I have no idea where it would save it being I am not a GCP customer, only 
Google Workspace. Workspace won't help because they say it's a GCP product but 
I am exporting from Workspace.

Can someone let me know the proper command to run on my local machine with 
Google's SDK to download this folder? I was able to start the download 
yesterday but it said there was an invalid character in the file names so it 
killed the export.

Appreciate any help!


On this page:

https://cloud.google.com/storage/docs/gsutil/commands/cp

I think the problem might be that you have an surplus backslash.

Does this work?

gsutil cp -r gs://takeout-export-myUniqueID .

This should copy recursively from gs://takeout-export-myUniqueID to the 
current folder.

--
https://mail.python.org/mailman/listinfo/python-list


Re: C API PyObject_Call segfaults with string

2022-02-09 Thread MRAB

On 2022-02-10 01:37, Jen Kris via Python-list wrote:

I'm using Python 3.8 so I tried your second choice:

pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);

but pSents is 0x0.  pSentMod and pListItem are valid pointers.


'PyObject_CallFunction' looks like a good one to use:

"""PyObject* PyObject_CallFunction(PyObject *callable, const char 
*format, ...)


Call a callable Python object callable, with a variable number of C 
arguments. The C arguments are described using a Py_BuildValue() style 
format string. The format can be NULL, indicating that no arguments are 
provided.

"""

[snip]

What I do is add comments to keep track of what objects I have 
references to at each point and whether they are new references or could 
be NULL.


For example:

pName = PyUnicode_FromString("nltk.corpus");
//> pName+?

This means that 'pName' contains a reference, '+' means that it's a new 
reference, and '?' means that it could be NULL (usually due to an 
exception, but not always) so I need to check it.


Continuing in this vein:

pModule = PyImport_Import(pName);
//> pName+? pModule+?

pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
//> pName+? pModule+? pSubMod+?
pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
//> pName+? pModule+? pSubMod+? pFidMod+?
pSentMod = PyObject_GetAttrString(pSubMod, "sents");
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+?

pFileIds = PyObject_CallObject(pFidMod, 0);
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+?

pListItem = PyList_GetItem(pFileIds, listIndex);
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+? pListItem?

pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
//> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+? pListItem? pListStrE+?


As you can see, there's a lot of leaked references building up.

Note how after:

pListItem = PyList_GetItem(pFileIds, listIndex);

the addition is:

//> pListItem?

This means that 'pListItem' contains a borrowed (not new) reference, but 
could be NULL.


I find it easiest to DECREF as soon as I no longer need the reference 
and remove a name from the list as soon I no longer need it (and 
DECREFed where).


For example:

pName = PyUnicode_FromString("nltk.corpus");
//> pName+?
if (!pName)
goto error;
//> pName+
pModule = PyImport_Import(pName);
//> pName+ pModule+?
Py_DECREF(pName);
//> pModule+?
if (!pModule)
goto error;
//> pModule+

I find that doing this greatly reduces the chances of getting the 
reference counting wrong, and I can remove the comments once I've 
finished the function I'm writing.

--
https://mail.python.org/mailman/listinfo/python-list


Re: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'

2022-02-09 Thread MRAB

On 2022-02-09 12:45, Christian Gollwitzer wrote:

Am 09.02.22 um 08:46 schrieb NArshad:

When I enter data using Tkinter form in an Excel file when the excel file is 
closed there is no error but when I enter data using Tkinter form when the 
excel is already open following error comes:



PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'



What to do to correct this error? I have already searched on google search many 
times but no solution was found.


It's impossible. Excel locks the file deliberately when it is open, so
that you can't overwrite it from a different program. Otherwise, the
file could become inconsistent.

It's the same the other way too; you can't open the file in Excel while 
Python has it open.



The correct way to handle it in the GUI is to tell the user via a
message box that the file is open and can't be written.

An alternative to writing the file directly would be that you remote
control Excel; I think it provides a DDE API:
https://support.microsoft.com/en-us/office/dde-function-79e8b21c-2054-4b48-9ceb-d2cf38dc17f9


--
https://mail.python.org/mailman/listinfo/python-list


Re: Can't get iterator in the C API

2022-02-08 Thread MRAB

On 2022-02-09 01:12, Jen Kris via Python-list wrote:

I am using the Python C API to load the Gutenberg corpus from the nltk library 
and iterate through the sentences.  The Python code I am trying to replicate is:

from nltk.corpus import gutenberg
for i, fileid in enumerate(gutenberg.fileids()):
     sentences = gutenberg.sents(fileid)
     etc

where gutenberg.fileids is, of course, iterable.

I use the following C API code to import the module and get pointers:

int64_t Call_PyModule()
{
     PyObject *pModule, *pName, *pSubMod, *pFidMod, *pFidSeqIter,*pSentMod;

     pName = PyUnicode_FromString("nltk.corpus");
     pModule = PyImport_Import(pName);

     if (pModule == 0x0){
     PyErr_Print();
     return 1; }

     pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
     pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
     pSentMod = PyObject_GetAttrString(pSubMod, "sents");

     pFidIter = PyObject_GetIter(pFidMod);
     int ckseq_ok = PySeqIter_Check(pFidMod);
     pFidSeqIter  = PySeqIter_New(pFidMod);

     return 0;
}

pSubMod, pFidMod and pSentMod all return valid pointers, but the iterator lines 
return zero:

pFidIter = PyObject_GetIter(pFidMod);
int ckseq_ok = PySeqIter_Check(pFidMod);
pFidSeqIter  = PySeqIter_New(pFidMod);

So the C API thinks gutenberg.fileids is not iterable, but it is.  What am I 
doing wrong?

Look at your Python code. You have "gutenberg.fileids()", so the 
'fileids' attribute is not an iterable itself, but a method that you 
need to call to get the iterable.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Openning Python program

2022-02-06 Thread MRAB

On 2022-02-06 08:14, createkmontalb...@gmail.com wrote:

I cannot open python after downloading it keeps going to modify/uninstall
?? please help


It sounds like you're just re-running the installer.

The installer should've installed IDLE. Try that for editing programs.
--
https://mail.python.org/mailman/listinfo/python-list


Re: Puzzling behaviour of Py_IncRef

2022-01-25 Thread MRAB

On 2022-01-25 23:50, Tony Flury via Python-list wrote:


On 25/01/2022 22:28, Barry wrote:



On 25 Jan 2022, at 14:50, Tony Flury via Python-list  
wrote:



On 20/01/2022 23:12, Chris Angelico wrote:

On Fri, 21 Jan 2022 at 10:10, Greg Ewing  wrote:
On 20/01/22 12:09 am, Chris Angelico wrote:

At this point, the refcount has indeed been increased.


   return self;
  }

And then you say "my return value is this object".

So you're incrementing the refcount, then returning it without
incrementing the refcount. Your code is actually equivalent to "return
self".

Chris, you're not making any sense. This is C code, so there's no
way that "return x" can change the reference count of x.

Yeah, I wasn't clear there. It was equivalent to *the Python code*
"return self". My apologies.


  > The normal thing to do is to add a reference to whatever you're
  > returning. For instance, Py_RETURN_NONE will incref None and then
  > return it.
  >

The OP understands that this is not a normal thing to do. He's
trying to deliberately leak a reference for the purpose of diagnosing
a problem.

It would be interesting to see what the actual refcount is after
calling this function.

After calling this without a double increment in the function the ref count is 
still only 1 - which means that the 'return self' effectively does a double 
decrement. My original message includes the Python code which calls this 
'leaky' function and you can see that despite the 'leaky POC' doing an 
increment ref count drops back to one after the return.

You are right this is not a normal thing to do, I am trying to understand the 
behaviour so my library does the correct thing in all cases - for example - 
imagine you have two nodes in a tree :

A --- > B

And your Python code has a named reference to A, and B also maintains a 
reference to A as it's parent.

In this case I would expect A to have a reference count of 2 (counted as 3 
through sys.getrefcount() - one for the named reference in the Python code - 
and one for the link from B back to A; I would also expect B to have a 
reference count here of 1 (just the reference from A - assuming nothing else 
referenced B).

My original code was incrementing the ref counts of A and B and then returning 
A. within the Python test code A had a refcount of 1 (and not the expected 2), 
but the refcount from B was correct as far as I could tell.



Yes, and that's why I was saying it would need a *second* incref.

ChrisA

Thank you to all of you for trying to help - I accept that the only way to make 
the code work is to do a 2nd increment.

I don't understand why doing a 'return self' would result in a double decrement 
- that seems utterly bizzare behaviour - it obviously works, but why.

The return self in C will not change the ref count.

I would suggest setting a break point in your code and stepping out of the 
function and seeing that python’s code does to the ref count.

Barry


Barry,

something odd is going on because the Python code isn't doing anything
that would cause the reference count to go from 3 inside the C function
to 1 once the method call is complete.

As far as I know the only things that impact the reference counts are :

   * Increments due to assigning a new name or adding it to a container.
   * Increment due to passing the object to a function (since that binds
 a new name)
   * Decrements due to deletion of a name
   * Decrement due to going out of scope
   * Decrement due to being removed from a container.

None of those things are happening in the python code.

As posted in the original message - immediately before the call to the C
function/method sys.getrefcount reports the count to be 2 (meaning it is
actually a 1).

Inside the C function the ref count is incremented and the Py_REFCNT
macro reports the count as 3 inside the C function as expected (1 for
the name in the Python code, 1 for the argument as passed to the C
function, and 1 for the increment), so outside the function one would
expect the ref count to now be 2 (since the reference caused by calling
the function is then reversed).

However - Immediately outside the C function and back in the Python code
sys.getrefcount reports the count to be 2 again - meaning it is now
really 1. So that means that the refcount has been decremented twice
in-between the return of the C function and the execution of the
immediate next python statement. I understand one of those decrements -
the parameter's ref count is incremented on the way in so the same
object is decremented on the way out (so that calls don't leak
references) but I don't understand where the second decrement is coming
from.

Again there is nothing in the Python code that would cause that
decrement - the decrement behavior is in the Python runtime.


The function returns a result, an object.

The calling code is discarding the result, so it's being DECREFed.

For example:

def foo():
return Node()

returns a new node, so its 

Re: keep getting a syntax error on the very first program I am running

2022-01-14 Thread MRAB

On 2022-01-15 01:12, Bob Griffin wrote:

I am running this program and keep getting this error.  Is this normal?



Invalid syntax.  Perhaps you forgot a comma?



Also the t in tags is highlighted.



I even tried different versions of Python also.



Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license()" for more information.

print("Hello World")

Hello World

input("\n\nPress the enter key to exit.")

I get an error only if I use the entirety of it as a program. Is that 
what you're doing, perhaps?


It's not actually a program, but the output from an interactive session.

You start IDLE (I'm assuming that what you're using) and it prints:

Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC 
v.1929 64

bit (AMD64)] on win32

and then shows a prompt.

At the prompt you type:

print("Hello World")

and it print the response:

Hello World

and so on.
--
https://mail.python.org/mailman/listinfo/python-list


Re: Using PIP in Python 3.10 on Windows 10

2022-01-14 Thread MRAB

On 2022-01-14 17:40, Jonathan Gossage wrote:

I have installed Python 3.10.1 on Windows 10 using the recommended Windows
Installer. When I try to access PIP from the command line, I get the
following result, even though Python itself is accessible.


C:\Users\jgoss>python
Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

quit()


C:\Users\jgoss>pip install pip --upgrade
Fatal error in launcher: Unable to create process using '"C:\Program
Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
install pip --upgrade': The system cannot find the file specified.
During the installation, I chose to install Python in a non-default
location and to set the Environment variables.
  The result of this attempt is shown below:


C:\Users\jgoss>python
Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

quit()


C:\Users\jgoss>pip install pip --upgrade
Fatal error in launcher: Unable to create process using '"C:\Program
Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
install pip --upgrade': The system cannot find the file specified.

It looks as if the launcher is expecting to find Python installed at
c:\Program Files\Python3.10 whereas it has actually been installed at
D:\Users\jgoss\AppData\local\python\python3.10. It seems that the launcher
has not been updated to the latest installation location for python and
that it also needs to handle a non-default install location. The same
problem occurs if I take the install option to install to the default
location.
Is there any workaround as PIP is essential to my environment?

These days it's recommended that you use the Python Launcher and the pip 
module:


py -m pip install pip --upgrade
--
https://mail.python.org/mailman/listinfo/python-list


Re: What to write or search on github to get the code for what is written below:

2022-01-12 Thread MRAB

On 2022-01-11 06:31, NArshad wrote:

-“How are the relevant cells identified in the spreadsheet?”
The column headings are:
BOOK_NAME
BOOK_AUTHOR
BOOK_ISBN
TOTAL_COPIES
COPIES_LEFT
BORROWER’S_NAME
ISSUE_DATE
RETURN_DATE


-“It's often the case that the cells on the first row contain text as column 
labels.”

These I have written above.


-“If that's what you have in your spreadsheet, then read the cells on the first 
row for the column labels and put them in a dict to map from column label to 
column number.”

This written above I do not understand how to code.


Well, you know how to read the contents of a cell, and how to put items 
into a dict (the key will be the cell contents and the value will be the 
column number).


The column numbers will go from 1 to sheet.last_column, although some of 
them might be empty (their value will be None), which you can remove 
from the dict afterwards.

--
https://mail.python.org/mailman/listinfo/python-list


Re: What to write or search on github to get the code for what is written below:

2022-01-10 Thread MRAB

On 2022-01-10 16:39, NArshad wrote:


Using openpyxl is pretty straightforward:


from openpyxl import load_workbook
wb = load_workbook(spreadsheet_path)
sheet = wb.active

# Reading the values in cells:
print('Cell A1 contains', sheet['A1'].value)
print('Cell A2 contains', sheet['A2'].value)
print('Cell B1 contains', sheet['B1'].value)

# Alternatively:
print('Cell A1 contains', sheet.cell(1, 1).value)
print('Cell A2 contains', sheet.cell(1, 2).value)
print('Cell B1 contains', sheet.cell(2, 1).value)

"""
The cell numbers (A1 or A2 or A3 or A4……… ) are not fixed they can be any. I 
don't know what the cell number is going to be that's what the problem is.

The user is going to enter the book name in an HTML form present on a website 
then it will be checked whether the book user has entered is present or not in 
the Excel file. If the book is present in the book bank and the user requires 
that book then one will be issued to the user and the total number of books 
will be reduced by 1 (one) and the user or borrower’s name will be entered in 
the Excel’s table row in which the book name is present separated by a comma by 
other borrower names. The borrower's name can be more than one because more 
than one copies of the book are there as these are the books that are taught in 
schools.
"""
  

[snip]
How are the relevant cells identified in the spreadsheet?

It's often the case that the cells on the first row contain text as 
column labels. If that's what you have in your spreadsheet, then read 
the cells on the first row for the column labels and put them in a dict 
to map from column label to column number.

--
https://mail.python.org/mailman/listinfo/python-list


Re: What to write or search on github to get the code for what is written below:

2022-01-09 Thread MRAB

On 2022-01-09 07:04, NArshad wrote:

On Friday, 7 January 2022 at 02:59:17 UTC+5, alister wrote:
On Thu, 6 Jan 2022 10:55:30 -0800 (PST), NArshad wrote: 

> All this is going to be in python’s flask and HTML only 
> 
> 1. First, I have to check in the Excel sheet or table whether the book 
> user has entered is present in the book bank or not. 
>
Excel is the wrong application for storing this data - it should be in a 
database
> 2. If a book is present and the quantity of the required book is greater 
> than 0 (COPIES_LEFT column in excel file) and if the user wants the 
> book, it will be assigned to the user which he will take from the book 
> bank physically. When COPIES_LEFT will is less than or equal to 0 the 
> message will be “Book finished or not present”.

Again Excel is not the correct application for processing this data
> 
> 3. The quantity of the book in the Excel file will be reduced by 1 in 
> the COPIES_LEFT column and the name of the borrower or user will be 
> entered/added in the Excel file table or sheet already made and the 
> column name is BORROWER’S NAME. 
>

Database!
> 4. The borrower’s or user name can be more than one so they will be 
> separated with a comma in the Excel file BORROWER’S NAME column. 
>

Database
> 
> - All functions mentioned above are to be deployed on the website 
> pythonhow.com so make according to 
> https://pythonhow.com/python-tutorial/flask/web-development-with-python- 
and-flask/ 
> 
> - Do you know any other websites to deploy a python web application?? 
> 
> - No time to switch from Excel to anywhere else. Please do not make any 
> changes to the Excel file. 
> 
> - Tutorials and repositories of the web for such problems are also 
> required. The same is required for python (flask, Django...) also.
Sorry did not spot that this was a homework assignment 
data should still be imported into a DB (a trivial task) It can be 
exported back to a compatible format just as easily if hard copy output is 
required 



As written no time to switch from excel to anywhere else but if a certain 
database is required to make changes in Excel’s cell values then which database 
to use (example Access or what) and after the right selection of the database, 
how to import data to a database and then export back to a compatible format 
that is to Excel cells….

I mean how to do this written below:

“data should still be imported into a DB (a trivial task) it can be exported 
back to a compatible format just as easily if hard copy output is required”

The reason I have written:

“What to write on GitHub or on google search to find the necessary code to 
start with? “

I will also be requiring a code to start with just as most people do. The same 
is the case with tutorials.

This is not homework and I have been checking openpyxl for the last one or two 
months to find what is required by me when I found nothing of what is required 
by me then posted on this google group.



Using openpyxl is pretty straightforward:


from openpyxl import load_workbook
wb = load_workbook(spreadsheet_path)
sheet = wb.active

# Reading the values in cells:
print('Cell A1 contains', sheet['A1'].value)
print('Cell A2 contains', sheet['A2'].value)
print('Cell B1 contains', sheet['B1'].value)

# Alternatively:
print('Cell A1 contains', sheet.cell(1, 1).value)
print('Cell A2 contains', sheet.cell(1, 2).value)
print('Cell B1 contains', sheet.cell(2, 1).value)

# Changing the value in a cell:
sheet.cell(1, 20).value = 'TEST'

# A value of None means that the cell is empty.

wb.save(spreadsheet_path)
wb.close()
--
https://mail.python.org/mailman/listinfo/python-list


Re: What to write or search on github to get the code for what is written below:

2022-01-07 Thread MRAB

On 2022-01-06 18:55, NArshad wrote:

All this is going to be in python’s flask and HTML only

1. First, I have to check in the Excel sheet or table whether the book user has 
entered is present in the book bank or not.

2. If a book is present and the quantity of the required book is greater than 0 
(COPIES_LEFT column in excel file) and if the user wants the book, it will be 
assigned to the user which he will take from the book bank physically. When 
COPIES_LEFT will is less than or equal to 0 the message will be “Book finished 
or not present”.

3. The quantity of the book in the Excel file will be reduced by 1 in the 
COPIES_LEFT column and the name of the borrower or user will be entered/added 
in the Excel file table or sheet already made and the column name is BORROWER’S 
NAME.

4. The borrower’s or user name can be more than one so they will be separated 
with a comma in the Excel file BORROWER’S NAME column.


- All functions mentioned above are to be deployed on the website pythonhow.com 
so make according to 
https://pythonhow.com/python-tutorial/flask/web-development-with-python-and-flask/

- Do you know any other websites to deploy a python web application??

- No time to switch from Excel to anywhere else. Please do not make any changes 
to the Excel file.

- Tutorials and repositories of the web for such problems are also required. 
The same is required for python (flask, Django...) also.

As someone else has already pointed, Excel is the wrong way to do it, 
yet it remains a popular method.


Given that, you can read Excel spreadsheets with 'openpyxl', which is on 
PyPI.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Shapely Polygon creating empty polygon

2022-01-04 Thread MRAB

On 2022-01-04 22:57, Israel Brewster wrote:

I’m running into an issue with shapely that is baffling me. Perhaps someone 
here can help out?

When running shapely directly from a python 3.8 interpreter, it works as 
expected:


import shapely
shapely.__version__

'1.8.0'

from shapely.geometry import Polygon
bounds = [-164.29635821669632, 54.64251856269729, -163.7631779798799, 
54.845450778742546]
print(Polygon.from_bounds(*bounds))

POLYGON ((-164.2963582166963 54.64251856269729, -164.2963582166963 
54.84545077874255, -163.7631779798799 54.84545077874255, -163.7631779798799 
54.64251856269729, -164.2963582166963 54.64251856269729))

However, if I put this exact same code into my Flask app (currently running 
under the Flask development environment) as part of handling a request, I get 
an empty polygon:


   import shapely
   print(shapely.__version__)
   from shapely.geometry import Polygon
   print(Polygon.from_bounds(*bounds))


In that piece of code you didn't set 'bounds'. It might be worth setting 
it or at least printing it out to double-check.


Output:

1.8.0
POLYGON EMPTY

In fact, *any* attempt to create a polygon gives the same result:

test = Polygon(((1, 1), (2, 1), (2, 2)))
print(test)

POLYGON EMPTY

Did you try that piece of code both in Flask and from the command line, 
again to double-check.



What am I missing here? Why doesn’t it work as part of a Flask request call?


--
https://mail.python.org/mailman/listinfo/python-list


Re: builtins.TypeError: catching classes that do not inherit from BaseException is not allowed

2021-12-30 Thread MRAB

On 2021-12-30 23:27, hongy...@gmail.com wrote:

On Friday, December 31, 2021 at 7:04:24 AM UTC+8, Chris Angelico wrote:
Neither of these wants to be recursive, and writing them recursively 
pollutes the function signature with parameters that really exist just 
to be local variables. Passing an accumulator down is a terrible way 
to demonstrate the beauty of recursion - it instead shows off how you 
can shoehorn anything into recursion and make it worse in the process. 


Then what cases/scenarios can demonstrate the beauty of recursion?


Walking a tree.
--
https://mail.python.org/mailman/listinfo/python-list


Re: problem reading a CSV file

2021-12-13 Thread MRAB

On 2021-12-12 23:37, Larry Warner wrote:

Win 10, Chrome, Python 3.10.1
New at python
error on open statement

Probably simple error but I do not see it.

The program is a python example with the file name being changed.  I want
to experiment with changing the literal file name in the open statement to
a variable name later.

It's difficult to say what the problem is when you haven't given us the 
code!

--
https://mail.python.org/mailman/listinfo/python-list


Re: Short, perfect program to read sentences of webpage

2021-12-08 Thread MRAB

On 2021-12-08 23:17, Stefan Ram wrote:

Cameron Simpson  writes:
Instead, consider the \b (word boundary) and \w (word character) 
markers, which will let you break strings up, and then maybe test the 
results with str.isupper().


   Thanks for your comments, most or all of them are
   valid, and I will try to take them into account!

   Regexps might have their disadvantages, but when I use them,
   it is clearer for me to do all the matching with regexps
   instead of mixing them with Python calls like str.isupper.
   Therefore, it is helpful for me to have a regexp to match
   upper and lower case characters separately. Some regexp
   dialects support "\p{Lu}" and "\p{Ll}" for this.

If you want "\p{Lu}" and "\p{Ll}", have a look at the 'regex' module on 
PyPI:


https://pypi.org/project/regex/

[snip]
--
https://mail.python.org/mailman/listinfo/python-list


Re: Short, perfect program to read sentences of webpage

2021-12-08 Thread MRAB

On 2021-12-08 19:39, Julius Hamilton wrote:

Hey,

This is something I have been working on for a very long time. It’s one of
the reasons I got into programming at all. I’d really appreciate if people
could input some advice on this.

This is a really simple program which extracts the text from webpages and
displays them one sentence at a time. It’s meant to help you study dense
material, especially documentation, with much more focus and comprehension.
I actually hope it can be of help to people who have difficulty reading. I
know it’s been of use to me at least.

This is a minimally acceptable way to pull it off currently:

deepreader.py:

import sys
import requests
import html2text
import nltk

url = sys.argv[1]

# Get the html, pull out the text, and sentence-segment it in one line of
code

sentences = nltk.sent_tokenize(html2text.html2text(requests.get(url).text))

# Activate an elementary reader interface for the text

for index, sentence in enumerate(sentences):

   # Print the sentence
   print(“\n” + str(index) + “/“ + str(len(sentences)) + “: “ + sentence +
“\n”)


You can shorten that with format strings:

print("\n{}/{}: {}\n".format(index, len(sentences), sentence))

or even:

print(f"\n{index}/{len(sentences)}: {sentence}\n")


   # Wait for user key-press
   x = input(“\n> “)


EOF



That’s it.

A lot of refining is possible, and I’d really like to see how some more
experienced people might handle it.

1. The HTML extraction is not perfect. It doesn’t produce as clean text as
I would like. Sometimes random links or tags get left in there. And the
sentences are sometimes randomly broken by newlines.

2. Neither is the segmentation perfect. I am currently researching
developing an optimal segmenter with tools from Spacy.

Brevity is greatly valued. I mean, anyone who can make the program more
perfect, that’s hugely appreciated. But if someone can do it in very few
lines of code, that’s also appreciated.

Thanks very much,
Julius



--
https://mail.python.org/mailman/listinfo/python-list


Re: Embedding Python crash on PyTuple_New

2021-11-24 Thread MRAB

On 2021-11-24 07:59, Arnaud Loonstra wrote:


On 24-11-2021 01:46, MRAB wrote:

On 2021-11-23 20:25, Arnaud Loonstra wrote:

On 23-11-2021 18:31, MRAB wrote:

On 2021-11-23 16:04, Arnaud Loonstra wrote:

On 23-11-2021 16:37, MRAB wrote:

On 2021-11-23 15:17, MRAB wrote:

On 2021-11-23 14:44, Arnaud Loonstra wrote:

On 23-11-2021 15:34, MRAB wrote:

On 2021-11-23 12:07, Arnaud Loonstra wrote:

Hi,

I've got Python embedded successfully in a program up until now 
as I'm
now running into weird GC related segfaults. I'm currently 
trying to

debug this but my understanding of CPython limits me here.

I'm creating a Tuple in C but it crashes on creating it after a 
while.

It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is 
cleaning up
stuff completely unrelated to the allocation of the new tuple? 
How can I

troubleshoot this?

I've got CPython compiled with  --with-valgrind --without-pymalloc
--with-pydebug

In C I'm creating a tuple with the following method:

static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
  assert(self);
  assert(oscmsg);
  char *format = zosc_format(oscmsg);

  PyObject *rettuple = PyTuple_New((Py_ssize_t) 
strlen(format) );


It segfaults here (frame 16) after 320 times (consistently)


1   __GI_raise raise.c  49   0x772c4e71
2   __GI_abort abort.c  79   0x772ae536
3   fatal_error    pylifecycle.c    2183 0x77d84b4f
4   Py_FatalError  pylifecycle.c    2193 0x77d878b2
5   _PyObject_AssertFailed object.c 2200 0x77c93cf2
6   visit_decref   gcmodule.c   378  0x77dadfd5
7   tupletraverse  tupleobject.c    623  0x77ca3e81
8   subtract_refs  gcmodule.c   406  0x77dad340
9   collect    gcmodule.c   1054 0x77dae838
10  collect_with_callback  gcmodule.c   1240 0x77daf17b
11  collect_generations    gcmodule.c   1262 0x77daf3f6
12  _PyObject_GC_Alloc gcmodule.c   1977 0x77daf4f2
13  _PyObject_GC_Malloc    gcmodule.c   1987 0x77dafebc
14  _PyObject_GC_NewVar    gcmodule.c   2016 0x77daffa5
15  PyTuple_New    tupleobject.c    118  0x77ca4da7
16  s_py_zosc_tuple    pythonactor.c    366  0x5568cc82
17  pythonactor_socket pythonactor.c    664  0x5568dac7
18  pythonactor_handle_msg pythonactor.c    862  0x5568e472
19  pythonactor_handler    pythonactor.c    828  0x5568e2e2
20  sphactor_actor_run sphactor_actor.c 855  0x558cb268
... 

Any pointer really appreciated.


[snip]



Basically, yes, but I won't be surprised if it was due to too few 
INCREFs or too many DECREFs somewhere.


https://github.com/hku-ect/gazebosc/blob/505b30c46bf3f78d188c3f575c80e294d3db7e5d/Actors/pythonactor.c#L286 



Incidentally, in s_py_zosc_tuple, you're not doing "assert(rc == 0);" 
after "after zosc_pop_float" or "zosc_pop_double".


Thanks for those pointers! I think your intuition is right. I might have
found the bugger. In s_py_zosc I call Py_DECREF on pAddress and pData.
However they are acquired by PyTuple_GetItem which returns a borrowed
reference. I think pAddress and pData are then also 'decrefed' when the
pReturn tuple which contains pAddress and pData is 'decrefed'?


Yes, members of a container are DECREFed when the container is destroyed.

It's bad practice for a function to DECREF its arguments unless the 
function's sole purpose is cleanup because the function won't know where 
the arguments came from.




I'm finding it out now. What strikes me was how hard it was to debug
this. I think it was caused because I INCREFed the return object. I
guess I did that to workaround the wrong DECREF data in the return
object. However that caused a hell to debug. I'm really curious what the
best practices are for debugging embedded CPython.

Thanks big time for your feedback!

What I do when writing the code is add comments showing what variables 
refer to an object at that point in the code, each suffixed with "+" if 
it owns a reference and/or "?" if it could be NULL.


Example 1:

//>
PyObject *my_tuple = PyTuple_New(count);
//> my_tuple+?
if (!my_tuple)
 goto error;
//> my_tuple+

"//>" means that there are no variables that point to an object.

"//> my_tuple+?" means that "my_tuple" points to an object and it owns a 
reference, but it might be NULL.


"//> my_tuple+" means that "my_tuple" points to an object and it owns a 
reference.


Example 2:

//>
PyObject *my_item = PyList_New(my_list, index);
//> my_tuple?
if (!my_tuple)
 goto error;
//> my_tuple

"//>" means that there are no variables that point to an object.

"//> my_tuple?" means that "my_tuple" points to an object, but it might 
be NULL.


"//> my_tuple" means that "my_tuple" points to an object.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Embedding Python crash on PyTuple_New

2021-11-23 Thread MRAB

On 2021-11-23 20:25, Arnaud Loonstra wrote:

On 23-11-2021 18:31, MRAB wrote:

On 2021-11-23 16:04, Arnaud Loonstra wrote:

On 23-11-2021 16:37, MRAB wrote:

On 2021-11-23 15:17, MRAB wrote:

On 2021-11-23 14:44, Arnaud Loonstra wrote:

On 23-11-2021 15:34, MRAB wrote:

On 2021-11-23 12:07, Arnaud Loonstra wrote:

Hi,

I've got Python embedded successfully in a program up until now 
as I'm

now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.

I'm creating a Tuple in C but it crashes on creating it after a 
while.

It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is 
cleaning up
stuff completely unrelated to the allocation of the new tuple? 
How can I

troubleshoot this?

I've got CPython compiled with  --with-valgrind --without-pymalloc
--with-pydebug

In C I'm creating a tuple with the following method:

static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
  assert(self);
  assert(oscmsg);
  char *format = zosc_format(oscmsg);

  PyObject *rettuple = PyTuple_New((Py_ssize_t) 
strlen(format) );


It segfaults here (frame 16) after 320 times (consistently)


1   __GI_raise raise.c  49   0x772c4e71
2   __GI_abort abort.c  79   0x772ae536
3   fatal_error    pylifecycle.c    2183 0x77d84b4f
4   Py_FatalError  pylifecycle.c    2193 0x77d878b2
5   _PyObject_AssertFailed object.c 2200 0x77c93cf2
6   visit_decref   gcmodule.c   378  0x77dadfd5
7   tupletraverse  tupleobject.c    623  0x77ca3e81
8   subtract_refs  gcmodule.c   406  0x77dad340
9   collect    gcmodule.c   1054 0x77dae838
10  collect_with_callback  gcmodule.c   1240 0x77daf17b
11  collect_generations    gcmodule.c   1262 0x77daf3f6
12  _PyObject_GC_Alloc gcmodule.c   1977 0x77daf4f2
13  _PyObject_GC_Malloc    gcmodule.c   1987 0x77dafebc
14  _PyObject_GC_NewVar    gcmodule.c   2016 0x77daffa5
15  PyTuple_New    tupleobject.c    118  0x77ca4da7
16  s_py_zosc_tuple    pythonactor.c    366  0x5568cc82
17  pythonactor_socket pythonactor.c    664  0x5568dac7
18  pythonactor_handle_msg pythonactor.c    862  0x5568e472
19  pythonactor_handler    pythonactor.c    828  0x5568e2e2
20  sphactor_actor_run sphactor_actor.c 855  0x558cb268
... 

Any pointer really appreciated.

You're creating a tuple that'll have the same number of members as 
the length of a string? That looks strange to me.


How are you setting the tuple's members?


It's from a serialisation format called OSC. The string describes the
type of bytes, every character is a type.

I'm creating the tuple as follows:

PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

Then I iterate the OSC message using the format string, (just showing
handling an int (i))

  char type = '0';
  Py_ssize_t pos = 0;
  const void *data =  zosc_first(oscmsg, );
  while(data)
  {
  switch (type)
  {
  case('i'):
  {
  int32_t val = 9;
  int rc = zosc_pop_int32(oscmsg, );
  assert(rc == 0);
  PyObject *o = PyLong_FromLong((long)val);
  assert( o );
  rc = PyTuple_SetItem(rettuple, pos, o);
  assert(rc == 0);
  break;
  }

Full code is here:

https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360 




Looking at that code, you have:

  PyObject *o = Py_BuildValue("s#", str, 1);

what I'd check is the type of the 1 that you're passing. Wouldn't the
compiler assume that it's an int?

The format string tells the function to expect a Py_ssize_t, but how
would the compiler know that?

Looking at https://www.mankier.com/3/zosc, it says for 'T' and 'F' 
"(no value required)", but you're doing:


 int rc = zosc_pop_bool(oscmsg, );

If no value is required, is there a bool there to be popped?


The value is only required to set a user provided boolean to the value
in the message. There's no boolean value encoded in the message, only
the T and F in the format string.

With regards to the Py_BuildValue("s#", str, 1);, that's a valid point.
I'll fix that. However in the segfaults I'm testing that code is not
touched.


You can use "C" as a format string for Py_BuildValue to convert a C int 
representing a character to a Python string.



I'm now testing different parts of the code to see if it runs stable.
I've found it runs stable if I do not process the returned tuple.

PyObject *pReturn = PyObject_CallMethod(self->pyinstance,
  "handleSocket", "sOsss",
  oscaddress,

Re: Embedding Python crash on PyTuple_New

2021-11-23 Thread MRAB

On 2021-11-23 17:31, MRAB wrote:

On 2021-11-23 16:04, Arnaud Loonstra wrote:

[snip]


I would turn my attention to s_py_zosc function but I'm not sure. Since
the errors are GC related it could caused anywhere?


Basically, yes, but I won't be surprised if it was due to too few
INCREFs or too many DECREFs somewhere.


https://github.com/hku-ect/gazebosc/blob/505b30c46bf3f78d188c3f575c80e294d3db7e5d/Actors/pythonactor.c#L286


Incidentally, in s_py_zosc_tuple, you're not doing "assert(rc == 0);"
after "after zosc_pop_float" or "zosc_pop_double".


Here's something interesting in s_py_zosc:

PyTypeObject* type = Py_TYPE(item);
const char * typename = _PyType_Name(type);
if (streq(typename, "c_int"))
{
zsys_warning("Unsupported ctypes.c_int until we find a 
way to access the value :S");

}
else
zsys_warning("unsupported python type");

Py_DECREF(type);

According to the docs, Py_TYPE returns a borrowed reference, but you're 
DECREFing it. If you call s_py_zosc enough times, the reference count of 
the type will drop to 0. I think that might be the problem.

--
https://mail.python.org/mailman/listinfo/python-list


Re: Embedding Python crash on PyTuple_New

2021-11-23 Thread MRAB

On 2021-11-23 16:04, Arnaud Loonstra wrote:

On 23-11-2021 16:37, MRAB wrote:

On 2021-11-23 15:17, MRAB wrote:

On 2021-11-23 14:44, Arnaud Loonstra wrote:

On 23-11-2021 15:34, MRAB wrote:

On 2021-11-23 12:07, Arnaud Loonstra wrote:

Hi,

I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.

I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How 
can I

troubleshoot this?

I've got CPython compiled with  --with-valgrind --without-pymalloc
--with-pydebug

In C I'm creating a tuple with the following method:

static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
  assert(self);
  assert(oscmsg);
  char *format = zosc_format(oscmsg);

  PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

It segfaults here (frame 16) after 320 times (consistently)


1   __GI_raise raise.c  49   0x772c4e71
2   __GI_abort abort.c  79   0x772ae536
3   fatal_error    pylifecycle.c    2183 0x77d84b4f
4   Py_FatalError  pylifecycle.c    2193 0x77d878b2
5   _PyObject_AssertFailed object.c 2200 0x77c93cf2
6   visit_decref   gcmodule.c   378  0x77dadfd5
7   tupletraverse  tupleobject.c    623  0x77ca3e81
8   subtract_refs  gcmodule.c   406  0x77dad340
9   collect    gcmodule.c   1054 0x77dae838
10  collect_with_callback  gcmodule.c   1240 0x77daf17b
11  collect_generations    gcmodule.c   1262 0x77daf3f6
12  _PyObject_GC_Alloc gcmodule.c   1977 0x77daf4f2
13  _PyObject_GC_Malloc    gcmodule.c   1987 0x77dafebc
14  _PyObject_GC_NewVar    gcmodule.c   2016 0x77daffa5
15  PyTuple_New    tupleobject.c    118  0x77ca4da7
16  s_py_zosc_tuple    pythonactor.c    366  0x5568cc82
17  pythonactor_socket pythonactor.c    664  0x5568dac7
18  pythonactor_handle_msg pythonactor.c    862  0x5568e472
19  pythonactor_handler    pythonactor.c    828  0x5568e2e2
20  sphactor_actor_run sphactor_actor.c 855  0x558cb268
... 

Any pointer really appreciated.

You're creating a tuple that'll have the same number of members as 
the length of a string? That looks strange to me.


How are you setting the tuple's members?


It's from a serialisation format called OSC. The string describes the
type of bytes, every character is a type.

I'm creating the tuple as follows:

PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

Then I iterate the OSC message using the format string, (just showing
handling an int (i))

  char type = '0';
  Py_ssize_t pos = 0;
  const void *data =  zosc_first(oscmsg, );
  while(data)
  {
  switch (type)
  {
  case('i'):
  {
  int32_t val = 9;
  int rc = zosc_pop_int32(oscmsg, );
  assert(rc == 0);
  PyObject *o = PyLong_FromLong((long)val);
  assert( o );
  rc = PyTuple_SetItem(rettuple, pos, o);
  assert(rc == 0);
  break;
  }

Full code is here:

https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360 




Looking at that code, you have:

  PyObject *o = Py_BuildValue("s#", str, 1);

what I'd check is the type of the 1 that you're passing. Wouldn't the
compiler assume that it's an int?

The format string tells the function to expect a Py_ssize_t, but how
would the compiler know that?

Looking at https://www.mankier.com/3/zosc, it says for 'T' and 'F' "(no 
value required)", but you're doing:


     int rc = zosc_pop_bool(oscmsg, );

If no value is required, is there a bool there to be popped?


The value is only required to set a user provided boolean to the value
in the message. There's no boolean value encoded in the message, only
the T and F in the format string.

With regards to the Py_BuildValue("s#", str, 1);, that's a valid point.
I'll fix that. However in the segfaults I'm testing that code is not
touched.


You can use "C" as a format string for Py_BuildValue to convert a C int 
representing a character to a Python string.



I'm now testing different parts of the code to see if it runs stable.
I've found it runs stable if I do not process the returned tuple.

PyObject *pReturn = PyObject_CallMethod(self->pyinstance,
  "handleSocket", "sOsss",
  oscaddress,
  py_osctuple,
  ev->type, ev->name, strdup(ev->uuid)

Re: Embedding Python crash on PyTuple_New

2021-11-23 Thread MRAB

On 2021-11-23 15:17, MRAB wrote:

On 2021-11-23 14:44, Arnaud Loonstra wrote:

On 23-11-2021 15:34, MRAB wrote:

On 2021-11-23 12:07, Arnaud Loonstra wrote:

Hi,

I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.

I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How can I
troubleshoot this?

I've got CPython compiled with  --with-valgrind --without-pymalloc
--with-pydebug

In C I'm creating a tuple with the following method:

static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
  assert(self);
  assert(oscmsg);
  char *format = zosc_format(oscmsg);

  PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

It segfaults here (frame 16) after 320 times (consistently)


1   __GI_raise raise.c  49   0x772c4e71
2   __GI_abort abort.c  79   0x772ae536
3   fatal_error    pylifecycle.c    2183 0x77d84b4f
4   Py_FatalError  pylifecycle.c    2193 0x77d878b2
5   _PyObject_AssertFailed object.c 2200 0x77c93cf2
6   visit_decref   gcmodule.c   378  0x77dadfd5
7   tupletraverse  tupleobject.c    623  0x77ca3e81
8   subtract_refs  gcmodule.c   406  0x77dad340
9   collect    gcmodule.c   1054 0x77dae838
10  collect_with_callback  gcmodule.c   1240 0x77daf17b
11  collect_generations    gcmodule.c   1262 0x77daf3f6
12  _PyObject_GC_Alloc gcmodule.c   1977 0x77daf4f2
13  _PyObject_GC_Malloc    gcmodule.c   1987 0x77dafebc
14  _PyObject_GC_NewVar    gcmodule.c   2016 0x77daffa5
15  PyTuple_New    tupleobject.c    118  0x77ca4da7
16  s_py_zosc_tuple    pythonactor.c    366  0x5568cc82
17  pythonactor_socket pythonactor.c    664  0x5568dac7
18  pythonactor_handle_msg pythonactor.c    862  0x5568e472
19  pythonactor_handler    pythonactor.c    828  0x5568e2e2
20  sphactor_actor_run sphactor_actor.c 855  0x558cb268
... 

Any pointer really appreciated.

You're creating a tuple that'll have the same number of members as the 
length of a string? That looks strange to me.


How are you setting the tuple's members?


It's from a serialisation format called OSC. The string describes the
type of bytes, every character is a type.

I'm creating the tuple as follows:

PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

Then I iterate the OSC message using the format string, (just showing
handling an int (i))

  char type = '0';
  Py_ssize_t pos = 0;
  const void *data =  zosc_first(oscmsg, );
  while(data)
  {
  switch (type)
  {
  case('i'):
  {
  int32_t val = 9;
  int rc = zosc_pop_int32(oscmsg, );
  assert(rc == 0);
  PyObject *o = PyLong_FromLong((long)val);
  assert( o );
  rc = PyTuple_SetItem(rettuple, pos, o);
  assert(rc == 0);
  break;
  }

Full code is here:

https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360


Looking at that code, you have:

  PyObject *o = Py_BuildValue("s#", str, 1);

what I'd check is the type of the 1 that you're passing. Wouldn't the
compiler assume that it's an int?

The format string tells the function to expect a Py_ssize_t, but how
would the compiler know that?

Looking at https://www.mankier.com/3/zosc, it says for 'T' and 'F' "(no 
value required)", but you're doing:


int rc = zosc_pop_bool(oscmsg, );

If no value is required, is there a bool there to be popped?
--
https://mail.python.org/mailman/listinfo/python-list


Re: Embedding Python crash on PyTuple_New

2021-11-23 Thread MRAB

On 2021-11-23 14:44, Arnaud Loonstra wrote:

On 23-11-2021 15:34, MRAB wrote:

On 2021-11-23 12:07, Arnaud Loonstra wrote:

Hi,

I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.

I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How can I
troubleshoot this?

I've got CPython compiled with  --with-valgrind --without-pymalloc
--with-pydebug

In C I'm creating a tuple with the following method:

static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
  assert(self);
  assert(oscmsg);
  char *format = zosc_format(oscmsg);

  PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

It segfaults here (frame 16) after 320 times (consistently)


1   __GI_raise raise.c  49   0x772c4e71
2   __GI_abort abort.c  79   0x772ae536
3   fatal_error    pylifecycle.c    2183 0x77d84b4f
4   Py_FatalError  pylifecycle.c    2193 0x77d878b2
5   _PyObject_AssertFailed object.c 2200 0x77c93cf2
6   visit_decref   gcmodule.c   378  0x77dadfd5
7   tupletraverse  tupleobject.c    623  0x77ca3e81
8   subtract_refs  gcmodule.c   406  0x77dad340
9   collect    gcmodule.c   1054 0x77dae838
10  collect_with_callback  gcmodule.c   1240 0x77daf17b
11  collect_generations    gcmodule.c   1262 0x77daf3f6
12  _PyObject_GC_Alloc gcmodule.c   1977 0x77daf4f2
13  _PyObject_GC_Malloc    gcmodule.c   1987 0x77dafebc
14  _PyObject_GC_NewVar    gcmodule.c   2016 0x77daffa5
15  PyTuple_New    tupleobject.c    118  0x77ca4da7
16  s_py_zosc_tuple    pythonactor.c    366  0x5568cc82
17  pythonactor_socket pythonactor.c    664  0x5568dac7
18  pythonactor_handle_msg pythonactor.c    862  0x5568e472
19  pythonactor_handler    pythonactor.c    828  0x5568e2e2
20  sphactor_actor_run sphactor_actor.c 855  0x558cb268
... 

Any pointer really appreciated.

You're creating a tuple that'll have the same number of members as the 
length of a string? That looks strange to me.


How are you setting the tuple's members?


It's from a serialisation format called OSC. The string describes the
type of bytes, every character is a type.

I'm creating the tuple as follows:

PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

Then I iterate the OSC message using the format string, (just showing
handling an int (i))

  char type = '0';
  Py_ssize_t pos = 0;
  const void *data =  zosc_first(oscmsg, );
  while(data)
  {
  switch (type)
  {
  case('i'):
  {
  int32_t val = 9;
  int rc = zosc_pop_int32(oscmsg, );
  assert(rc == 0);
  PyObject *o = PyLong_FromLong((long)val);
  assert( o );
  rc = PyTuple_SetItem(rettuple, pos, o);
  assert(rc == 0);
  break;
  }

Full code is here:

https://github.com/hku-ect/gazebosc/blob/822452dfa27201db274d37ce09e835d98fe500b2/Actors/pythonactor.c#L360


Looking at that code, you have:

PyObject *o = Py_BuildValue("s#", str, 1);

what I'd check is the type of the 1 that you're passing. Wouldn't the 
compiler assume that it's an int?


The format string tells the function to expect a Py_ssize_t, but how 
would the compiler know that?

--
https://mail.python.org/mailman/listinfo/python-list


Re: Embedding Python crash on PyTuple_New

2021-11-23 Thread MRAB

On 2021-11-23 12:07, Arnaud Loonstra wrote:

Hi,

I've got Python embedded successfully in a program up until now as I'm
now running into weird GC related segfaults. I'm currently trying to
debug this but my understanding of CPython limits me here.

I'm creating a Tuple in C but it crashes on creating it after a while.
It doesn't make sense which makes me wonder something else must be
happening? Could be it just crashes here because the GC is cleaning up
stuff completely unrelated to the allocation of the new tuple? How can I
troubleshoot this?

I've got CPython compiled with  --with-valgrind --without-pymalloc
--with-pydebug

In C I'm creating a tuple with the following method:

static PyObject *
s_py_zosc_tuple(pythonactor_t *self, zosc_t *oscmsg)
{
  assert(self);
  assert(oscmsg);
  char *format = zosc_format(oscmsg);

  PyObject *rettuple = PyTuple_New((Py_ssize_t) strlen(format) );

It segfaults here (frame 16) after 320 times (consistently)


1   __GI_raise raise.c  49   0x772c4e71
2   __GI_abort abort.c  79   0x772ae536
3   fatal_errorpylifecycle.c2183 0x77d84b4f
4   Py_FatalError  pylifecycle.c2193 0x77d878b2
5   _PyObject_AssertFailed object.c 2200 0x77c93cf2
6   visit_decref   gcmodule.c   378  0x77dadfd5
7   tupletraverse  tupleobject.c623  0x77ca3e81
8   subtract_refs  gcmodule.c   406  0x77dad340
9   collectgcmodule.c   1054 0x77dae838
10  collect_with_callback  gcmodule.c   1240 0x77daf17b
11  collect_generationsgcmodule.c   1262 0x77daf3f6
12  _PyObject_GC_Alloc gcmodule.c   1977 0x77daf4f2
13  _PyObject_GC_Mallocgcmodule.c   1987 0x77dafebc
14  _PyObject_GC_NewVargcmodule.c   2016 0x77daffa5
15  PyTuple_Newtupleobject.c118  0x77ca4da7
16  s_py_zosc_tuplepythonactor.c366  0x5568cc82
17  pythonactor_socket pythonactor.c664  0x5568dac7
18  pythonactor_handle_msg pythonactor.c862  0x5568e472
19  pythonactor_handlerpythonactor.c828  0x5568e2e2
20  sphactor_actor_run sphactor_actor.c 855  0x558cb268
... 

Any pointer really appreciated.

You're creating a tuple that'll have the same number of members as the 
length of a string? That looks strange to me.


How are you setting the tuple's members?
--
https://mail.python.org/mailman/listinfo/python-list


Re: doubt About import machine

2021-11-21 Thread MRAB

On 2021-11-21 18:36, Daniel Eduardo Almeida Correa wrote:

Hello, I'm trying to use the machine library in python 3.10 version, but I
can't import it with the pip install machine, could you tell me a way to
solve it  or a python version compatible with the library? Thank you a lot
for your answer.

Are you talking about the "machine" module of MicroPython? I believe 
that's specific to MicroPython.

--
https://mail.python.org/mailman/listinfo/python-list


Re: get_axes not present?

2021-11-21 Thread MRAB

On 2021-11-21 16:39, Mahmood Naderan via Python-list wrote:

The best way to get
assistance here on the list is to create a minimal, self-contained,
run-able, example program that you can post in its entirety here that
demonstrates the issue.



I created a sample code with input. Since the code processes a csv file to 
group input rows, I also included those in this minimal code but those 
preprocesses are not buggy. In this sample code, I used print() to print 
necessary information. The error exists in the plot function. I tested the 
dictionary build before that and it is fine.


Code is available at https://pastebin.com/giAnjJDV  and the input file 
(test.batch.csv) is available https://pastebin.com/Hdp4Wt9B

The run command is "python3 test.py". With the versions specified in my system, 
here is the full output:





$ python3 test.py
Reading file...
matplotlib version =  3.3.4
pandas version =  1.2.3
sys version sys.version_info(major=3, minor=8, micro=10, releaselevel='final', 
serial=0)
Original dictionary =  {'dummy': Value
M1  0
M2  0
M3  0, 'K1::foo(bar::z(x,u))':    Value  Value
0 10  2
1  5  2
2 10  2, 'K2::foo()':    Value
0 20
1 10
2 15, 'K3::foo(baar::y(z,u))':    Value
0 12
1 13
2 14, 'K3::foo(bar::y(z,u))':    Value
0  6
1  7
2  8}
New dictionary for plot =  {'dummy': Value
M1  0
M2  0
M3  0, 'K1::foo(bar::z(x,u))':    Value  Value
0 10  2
1  5  2
2 10  2, 'K3::foo(bar::y(z,u))':    Value
0  6
1  7
2  8}
Key is  K1::foo(bar::z(x,u))  -> df is Value  Value
0 10  2
1  5  2
2 10  2
axes= [ ]
axes[0]= AxesSubplot(0.125,0.53;0.775x0.35)
cnt= 1
row= 1    10
2 2
Name: 0, dtype: int64
Traceback (most recent call last):
   File "test.py", line 74, in 
     plot_kernels(my_dict2)
   File "test.py", line 52, in plot_kernels
     plot_dataframe(df, cnt, axes)
   File "test.py", line 36, in plot_dataframe
     ax1 = row.plot(label=cnt, ax=axes[0], marker='o')   # Line chart
   File 
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_core.py", 
line 955, in __call__
     return plot_backend.plot(data, kind=kind, **kwargs)
   File 
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/__init__.py",
 line 61, in plot
     plot_obj.generate()
   File 
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py",
 line 283, in generate
     self._adorn_subplots()
   File 
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py",
 line 483, in _adorn_subplots
     all_axes = self._get_subplots()
   File 
"/home/mahmood/.local/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py",
 line 903, in _get_subplots
     ax for ax in self.axes[0].get_figure().get_axes() if isinstance(ax, 
Subplot)
AttributeError: 'NoneType' object has no attribute 'get_axes'



I am pretty sure that there is a version mismatch because on a system with 
Pandas 1.3.3 the output should be like https://imgur.com/a/LZ9eAzl

Any feedback is appreciated.

I installed the latest pandas, although on Python 3.10, and the script 
worked without a problem.

--
https://mail.python.org/mailman/listinfo/python-list


Re: No right operator in tp_as_number?

2021-11-20 Thread MRAB

On 2021-11-20 17:45, Marco Sulla wrote:

I checked the documentation:
https://docs.python.org/3/c-api/typeobj.html#number-structs
and it seems that, in the Python C API, the right operators do not exist.
For example, there is nb_add, that in Python is __add__, but there's
no nb_right_add, that in Python is __radd__

Am I missing something?


A quick Google came up with this:

Python's __radd__ doesn't work for C-defined types
https://stackoverflow.com/questions/18794169/pythons-radd-doesnt-work-for-c-defined-types

It's about Python 2.7, but the principle is the same.
--
https://mail.python.org/mailman/listinfo/python-list


  1   2   3   4   5   6   7   8   9   10   >