[Python-announce] [RELEASE] pyspread 2.2.2
pyspread 2.2.2 == This is a bugfix release. Bug fixes: * pip install fixed by replacing deprecated setup.py option `requires` with `install_requires` * Shebang removed from desktop file About pyspread == Pyspread is a non-traditional spreadsheet that is based on and written in the programming language Python. The goal of pyspread is to be the most pythonic spreadsheet application. Pyspread is free software. It is released under the GPL v3. Project website: https://pyspread.gitlab.io/ Download page: https://pypi.org/project/pyspread/ Signature for tarball: https://gitlab.com/pyspread/downloads/-/raw/master/releases/pyspread-2.2.2.tar.gz.asc?inline=false Source code: https://gitlab.com/pyspread/pyspread Dependencies Mandatory: * Python (≥ 3.6) * numpy (>=1.1) * PyQt5 (≥ 5.10, requires PyQt5.Svg) * setuptools (>=40.0) * markdown2 (>= 2.3) Recommended: * matplotlib (>=1.1.1) * pyenchant (>=1.1) * pip (>=18) * python-dateutil (>= 2.7.0) * py-moneyed (>=2.0) * rpy2 (>=3.4) * plotnine (>=0.8) * libvoikko (>=4.3) * nuspell (>=5.1) * hspell (>= 1.4) * ggplot2 (>=3.4) For building the apidocs with Sphinx see apidocs/requirements.txt Enjoy Martin ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
Re: scipy.optimize.least_squares for more than one dimension?
Den 2023-06-30 skrev Martin Schöön : > Yesterday I wanted to move from optimize.leastsq to > least_squares. I have data depending on four variables > and want to fit a function in four variables to this > data. This works with leastsq but not with least_squares. > > Am I trying to do something least_squares is not capable > of? > > Disclaimer: I was burning midnight oil... > Problem solved. Yes, least_squares can, of course, handle multi-dimensional situations. Me burning midnight oil was the problem. I have been tinkering a bit with scipy.optimize.least_squares tonight. All simple examples I tried worked regardless of number of dimensions. I went back to my old code and found a couple of basic mistakes. Done. /Martin -- https://mail.python.org/mailman/listinfo/python-list
scipy.optimize.least_squares for more than one dimension?
Yesterday I wanted to move from optimize.leastsq to least_squares. I have data depending on four variables and want to fit a function in four variables to this data. This works with leastsq but not with least_squares. Am I trying to do something least_squares is not capable of? Disclaimer: I was burning midnight oil... TIA /Martin -- https://mail.python.org/mailman/listinfo/python-list
[Python-announce] [RELEASE] pyspread 2.2.1
pyspread 2.2 This is a bugfix release Note that missing libvoikko, nuspell and hspell may lead to warning messages if pyenchant is installed. However, pyspread is still functional as this affects only spellchecking. Bug fixes: * Chart dialog now correctly starts if rpy2 is not installed and if optional R modules are missing. * Deleting large selections is now significantly faster. * Pasting into large selections is now significantly faster. * QPen cache now is limited in size. About pyspread == Pyspread is a non-traditional spreadsheet that is based on and written in the programming language Python. The goal of pyspread is to be the most pythonic spreadsheet application. Pyspread is free software. It is released under the GPL v3. Project website: https://pyspread.gitlab.io/ Download page: https://pypi.org/project/pyspread/ Signature for tarball: https://gitlab.com/pyspread/downloads/-/raw/master/releases/pyspread-2.2.tar.gz.sig Source code: https://gitlab.com/pyspread/pyspread Dependencies Mandatory: * Python (≥ 3.6) * numpy (>=1.1) * PyQt5 (≥ 5.10, requires PyQt5.Svg) * setuptools (>=40.0) * markdown2 (>= 2.3) Recommended: * matplotlib (>=1.1.1) * pyenchant (>=1.1) * pip (>=18) * python-dateutil (>= 2.7.0) * py-moneyed (>=2.0) * rpy2 (>=3.4) * plotnine (>=0.8) * libvoikko (>=4.3) * nuspell (>=5.1) * hspell (>= 1.4) * ggplot2 (>=3.4) For building the apidocs with Sphinx see apidocs/requirements.txt Enjoy Martin ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [RELEASE] pyspread 2.2
pyspread 2.2 This release adds R charts via rpy2. Plotnine is also supported. Examples are available in the chart dialog. Note that the R packages graphics, lattice and ggplot2 are used for the examples. Bug fixes: * Thick lines are now antialiased About pyspread == Pyspread is a non-traditional spreadsheet that is based on and written in the programming language Python. The goal of pyspread is to be the most pythonic spreadsheet application. Pyspread is free software. It is released under the GPL v3. Project website: https://pyspread.gitlab.io/ Download page: https://pypi.org/project/pyspread/ Signature for tarball: https://gitlab.com/pyspread/downloads/-/raw/master/releases/pyspread-2.2.tar.gz.sig Source code: https://gitlab.com/pyspread/pyspread Dependencies Mandatory: * Python (≥ 3.6) * numpy (>=1.1) * PyQt5 (≥ 5.10, requires PyQt5.Svg) * setuptools (>=40.0) * markdown2 (>= 2.3) Recommended: * matplotlib (>=1.1.1) * pyenchant (>=1.1) * pip (>=18) * python-dateutil (>= 2.7.0) * py-moneyed (>=2.0) * rpy2 (>=3.4) * plotnine (>=0.8) For building the apidocs with Sphinx see apidocs/requirements.txt Enjoy Martin ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
Re: Problem with Matplotlib example
Den 2023-04-13 skrev MRAB : > On 2023-04-13 19:41, Martin Schöön wrote: >> Anyone had success running this example? >> https://tinyurl.com/yhhyc9r >> >> As far as I know I have an up-to-date matplotlib installed. Pip has >> nothing more modern to offer me. >> > All I can say is that it works for me! > > Python 3.10 and 3.11, matplotlib 3.6.1 and then 3.7.1 after updating it. > Thanks are due to both you and Thomas. Your replies put me on the right scent. I soon learned what I should have know since ages: pip reporting there is now newer version of a package does not mean there is *no newer* version. It means there is no newer version for the version of Python I use. In my case I am on Python 3.7. At work, where I am on Python 3.8, this matplotlib example works just fine. /Martin -- https://mail.python.org/mailman/listinfo/python-list
Problem with Matplotlib example
Anyone had success running this example? https://tinyurl.com/yhhyc9r When I try I get this error: "TypeError: __init__() got an unexpected keyword argument 'transform'" This is for the line "m = MarkerStyle(SUCESS_SYMBOLS[mood], transform=t)" Yes, I know, I could dive into the documentation myself but I hope some kind soul here will help out. As far as I know I have an up-to-date matplotlib installed. Pip has nothing more modern to offer me. TIA /Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: on the python paradox
On Mon, Dec 05, 2022 at 10:37:39PM -0300, Sabrina Almodóvar wrote: The Python Paradox Paul Graham August 2004 [SNIP] Hence what, for lack of a better name, I'll call the Python paradox: if a company chooses to write its software in a comparatively esoteric language, they'll be able to hire better programmers, because they'll attract only those who cared enough to learn it. And for programmers the paradox is even more pronounced: the language to learn, if you want to get a good job, is a language that people don't learn merely to get a job. [SNIP] I don't think that an esoteric language leads to better programmers. I know really good people that work mostly in assembly which by today standard would be considered "esoteric". They are really good at their field but they write shitty code in higher languages as python. That same goes for the other direction: I saw Ruby programmers writing C code and trust me, it didn't result in good quality code. I would be more inclined to think that a good programmer is not the one that knows an esoteric language but the one that can jump from one programming paradigm to another. And when I say "jump" I mean that he/she can understand the problem to solve, find the best tech stack to solve it and do it in an efficient manner using that tech stack correctly. It is in the "using that tech stack correctly" where some programmers that "think" they know languages A, B and C get it wrong. Just writing code that "compiles" and "it does not immediately crash" is not enough to say that "you are using the tech stack correctly". On Wed, Dec 07, 2022 at 10:58:09AM -0500, David Lowry-Duda wrote: On Mon, Dec 05, 2022 at 10:37:39PM -0300, Sabrina Almodóvar wrote: The Python Paradox Paul Graham August 2004 [SNIP] Hence what, for lack of a better name, I'll call the Python paradox: if a company chooses to write its software in a comparatively esoteric language, they'll be able to hire better programmers, because they'll attract only those who cared enough to learn it. And for programmers the paradox is even more pronounced: the language to learn, if you want to get a good job, is a language that people don't learn merely to get a job. [SNIP] I wonder what the appropriately esoteric language is today? We can sort of think of go/rust as esoteric versions of C/C++. But what would be the esoteric python? Perhaps Julia? I don't know of any large software projects happening in julia world that aren't essentially scientific computing libraries (but this is because *I* work mostly with scientific computing libraries and sometimes live under a rock). - DLD -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Which architectures to support in a CI like Travis?
I would depend on the project. In the crytoanalysis tool that I developing, "cryptonita", I just manipule bytes. Nothing that could depend on the distro so my CI picks one OS and run the tests there. Project: https://github.com/cryptonitas/cryptonita CI: https://github.com/cryptonitas/cryptonita/blob/master/.github/workflows/test.yml On the other extreme I have "byexample", a tool that takes the examples in your docs and run them as automated tests. It supports different languages (Python, Ruby, Java, ...) and it works using the interpreter of each languages. An there is the challenge for its CI. Because byexample highly depends on the version of the interpreter, the CI config tries a lot of different scenarios Project: https://byexamples.github.io/ CI: https://github.com/byexamples/byexample/blob/master/.github/workflows/test.yml I don't tests different distros but I should for some cases that I suspect that it could be differences in how some interpreters behave. An about OS, I'm planning to add MacOS to the CI because I know that some users had problems in the past in that platform because how byexample interacts with the terminal. So two projects, both in Python, but with two totally different dependencies on the environment where they run, so their CI are different. The two examples are using Gitlab actions but the same applies to TravisCI. Thanks, Martin. On Sun, Sep 18, 2022 at 09:46:45AM +, c.bu...@posteo.jp wrote: Hello, I am using TravisCI for my project on GitHub. The project is packaged for Debian, Ubuntu, Arch and several other distros. All this distros support multiple architectures and they have their own test machines to take care that all packages working on all archs. On my side (upstream) I wonder which arch I should "support" in my TravisCI configuration. I wan't to speed up travis and I want to save energy and carbon. I suspect that my Python code should run on much every platform that offers a Python interpreter. Of course there are edge cases. But they would be captured by the distros own test environments. So is there a good and objective reason to support multiple (and maybe) exotic platforms in a CI pipeline on upstream? Kind Christian -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple TCP proxy
On Wed, Jul 27, 2022 at 08:32:31PM +0200, Morten W. Petersen wrote: You're thinking of the backlog argument of listen? From my understanding, yes, when you set up the "accepter" socket (the one that you use to listen and accept new connections), you can define the length of the queue for incoming connections that are not accepted yet. This will be the equivalent of your SimpleQueue which basically puts a limits on how many incoming connections are "accepted" to do a real job. Using skt.listen(N) the incoming connections are put on hold by the OS while in your implementation are formally accepted but they are not allowed to do any meaningful work: they are put on the SimpleQueue and only when they are popped then they will work (send/recv data). The difference then between the OS and your impl is minimal. The only case that I can think is that on the clients' side it may exist a timeout for the acceptance of the connection so your proxy server will eagerly accept these connections so no timeout is possible(*) On a side note, you implementation is too thread-naive: it uses plain Python lists, integers and boolean variables which are not thread safe. It is a matter of time until your server will start behave weird. One option is that you use thread-safe objects. I'll encourage to read about thread-safety in general and then which sync mechanisms Python offers. Another option is to remove the SimpleQueue and the background function that allows a connection to be "active". If you think, the handlers are 99% independent except that you want to allow only N of them to progress (stablish and forward the connection) and when a handler finishes, another handler "waiting" is activated, "in a queue fashion" as you said. If you allow me to not have a strict queue discipline here, you can achieve the same results coordinating the handlers using semaphores. Once again, take this email as starting point for your own research. On a second side note, the use of handlers and threads is inefficient because while you have N active handlers sending/receiving data, because you are eagerly accepting new connections you will have much more handlers created and (if I'm not wrong), each will be a thread. A more efficient solution could be 1) accept as many connections as you can, saving the socket (not the handler) in the thread-safe queue. 2) have N threads in the background popping from the queue a socket and then doing the send/recv stuff. When the thread is done, the thread closes the socket and pops another from the queue. So the queue length will be the count of accepted connections but in any moment your proxy will not activate (forward) more than N connections. This idea is thread-safe, simpler, efficient and has the queue discipline (I leave aside the usefulness). I encourage you to take time to read about the different things mentioned as concurrency and thread-related stuff is not easy to master. Thanks, Martin. (*) make your proxy server slow enough and yes, you will get timeouts anyways. Well, STP will accept all connections, but can limit how many of the accepted connections that are active at any given time. So when I bombed it with hundreds of almost simultaneous connections, all of them were accepted, but only 25 were actively sending and receiving data at any given time. First come, first served. Regards, Morten On Wed, Jul 27, 2022 at 8:00 PM Chris Angelico wrote: On Thu, 28 Jul 2022 at 02:15, Morten W. Petersen wrote: > > Hi. > > I'd like to share with you a recent project, which is a simple TCP proxy > that can stand in front of a TCP server of some sort, queueing requests and > then allowing n number of connections to pass through at a time: How's this different from what the networking subsystem already does? When you listen, you can set a queue length. Can you elaborate? ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- I am https://leavingnorway.info Videos at https://www.youtube.com/user/TheBlogologue Twittering at http://twitter.com/blogologue Blogging at http://blogologue.com Playing music at https://soundcloud.com/morten-w-petersen Also playing music and podcasting here: http://www.mixcloud.com/morten-w-petersen/ On Google+ here https://plus.google.com/107781930037068750156 On Instagram at https://instagram.com/morphexx/ -- I am https://leavingnorway.info Videos at https://www.youtube.com/user/TheBlogologue Twittering at http://twitter.com/blogologue Blogging at http://blogologue.com Playing music at https://soundcloud.com/morten-w-petersen Also playing music and podcasting here: http://www.mixcloud.com/morten-w-petersen/ On Instagram at https://instagram.com/morphexx/ -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: exec() an locals() puzzle
I did a few tests
# test 1
def f():
i = 1
print(locals())
exec('y = i; print(y); print(locals())')
print(locals())
a = eval('y')
print(locals())
u = a
print(u)
f()
{'i': 1}
1
{'i': 1, 'y': 1}
{'i': 1, 'y': 1}
{'i': 1, 'y': 1, 'a': 1}
1
# test 2
def f():
i = 1
print(locals())
exec('y = i; print(y); print(locals())')
print(locals())
a = eval('y')
print(locals())
y = a
print(y)
f()
{'i': 1}
1
{'i': 1, 'y': 1}
{'i': 1}
Traceback (most recent call last):
NameError: name 'y' is not defined
So test 1 and 2 are the same except that the variable 'y' is not
present/present in the f's code.
When it is not present, exec() modifies the f's locals and adds an 'y'
to it but when the variable 'y' is present in the code (even if not
present in the locals()), exec() does not add any 'y' (and the next
eval() then fails)
The interesting part is that if the 'y' variable is in the f's code
*and* it is defined in the f's locals, no error occur but once again the
exec() does not modify f's locals:
# test 3
def f():
i = 1
y = 42
print(locals())
exec('y = i; print(y); print(locals())')
print(locals())
a = eval('y')
print(locals())
y = a
print(y)
f()
{'i': 1, 'y': 42}
1
{'i': 1, 'y': 1}
{'i': 1, 'y': 42}
{'i': 1, 'y': 42, 'a': 42}
42
Why does this happen? No idea.
I may be related with this:
# test 4
def f():
i = 1
print(locals())
exec('y = i; print(y); print(locals())')
print(locals())
print(y)
f()
Traceback (most recent call last):
NameError: name 'y' is not defined
Despite exec() adds the 'y' variable to f's locals, the variable is not
accessible/visible from f's code.
So, a few observations (by no means this is how the vm works):
1) each function has a set of variables defined by the code (let's call
this "code-defined locals" or "cdef-locals").
2) each function also has a set of "runtime locals" accessible from
locals().
3) exec() can add variables to locals() (runtime) set but it cannot add
any to cdef-locals.
4) locals() may be a superset of cdef-locals (but entries in cdef-locals
which value is still undefined are not shown in locals())
5) due rule 4, exec() cannot add a variable to locals() if it is already
present in the in cdef-locals.
6) when eval() runs, it uses locals() set for lookup
Perhaps rule 5 is to prevent exec() to modify any arbitrary variable of
the caller...
Anyways, nice food for our brains.
On Wed, Jul 20, 2022 at 04:56:02PM +, george trojan wrote:
I wish I could understand the following behaviour:
1. This works as I expect it to work:
def f():
i = 1
print(locals())
exec('y = i; print(y); print(locals())')
print(locals())
exec('y *= 2')
print('ok:', eval('y'))
f()
{'i': 1}
1
{'i': 1, 'y': 1}
{'i': 1, 'y': 1}
ok: 2
2. I can access the value of y with eval() too:
def f():
i = 1
print(locals())
exec('y = i; print(y); print(locals())')
print(locals())
u = eval('y')
print(u)
f()
{'i': 1}
1
{'i': 1, 'y': 1}
{'i': 1, 'y': 1}
1
3. When I change variable name u -> y, somehow locals() in the body of
the function loses an entry:
def f():
i = 1
print(locals())
exec('y = i; print(y); print(locals())')
print(locals())
y = eval('y')
print(y)
f()
{'i': 1}
1
{'i': 1, 'y': 1}
{'i': 1}
---NameError
Traceback (most recent call last)
Input In [1], in () 7 print(y) 8 # y
= eval('y') 9 #print('ok:', eval('y'))---> 10 f()
Input In [1], in f() 4 exec('y = i; print(y); print(locals())')
5 print(locals())> 6 y = eval('y') 7 print(y)
File :1, in
NameError: name 'y' is not defined1.
Another thing: within the first exec(), the print order seems
reversed. What is going on?
BTW, I am using python 3.8.13.
George
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: list indices must be integers or slices, not str
offtopic
If you want a pure-python but definitely a more hacky implementation,
you can play around with inspect.stack() and get the variables from
the outer frames.
# code:
x = 32
y = 42
printf("Hello x={x}, y={y}", x=27)
# output:
Hello x=27, y=42
The implementation of printf() was never released in PyPI (I guess I never saw
it as more than a challenge).
But the implementation is quite simple (I did a post about it):
https://book-of-gehn.github.io/articles/2021/07/11/Home-Made-Python-F-String.html
Thanks,
Martin.
On Wed, Jul 20, 2022 at 10:46:35AM -0600, Mats Wichmann wrote:
On 7/20/22 05:04, Frank Millman wrote:
I think the preferred style these days is f'{x[-1]}' which works."
Unfortunately the 'f' option does not work for me in this case, as I am
using a string object, not a string literal.
For that you could consider
https://pypi.org/project/f-yeah/
(straying a bit off thread subject by now, admittedly)
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Simple message passing system and thread safe message queue
Hi, I couldn't read your posts, every time I try to open one I'm redirected to an index page. I took a look at the smps project and I as far I understand it is a SSL client that sends messages to a server that implements a store of messages. I would suggest to remove the sleep() calls and as a challenge for you, make the server single-thread using asyncio and friends. Thanks, Martin. On Mon, Jul 18, 2022 at 06:31:28PM +0200, Morten W. Petersen wrote: Hi. I wrote a couple of blog posts as I had to create a message passing system, and these posts are here: http://blogologue.com/search?category=1658082823X26 Any comments or suggestions? Regards, Morten -- I am https://leavingnorway.info Videos at https://www.youtube.com/user/TheBlogologue Twittering at http://twitter.com/blogologue Blogging at http://blogologue.com Playing music at https://soundcloud.com/morten-w-petersen Also playing music and podcasting here: http://www.mixcloud.com/morten-w-petersen/ On Google+ here https://plus.google.com/107781930037068750156 On Instagram at https://instagram.com/morphexx/ -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: TENGO PROBLEMAS AL INSTALAR PYTHON
On Fri, Jul 08, 2022 at 04:15:35PM -0600, Mats Wichmann wrote: In addition... there is no "Python 10.0" ... Mmm, perhaps that's the problem :D @Angie Odette Lima Banguera, vamos a necesitar algun traceback o algo para guiarte. Podes tambien buscar en internet (youtube) q hay varios tutoriales para los primeros pasos. Good luck! -- https://mail.python.org/mailman/listinfo/python-list
Re: Filtering XArray Datasets?
Hi, I'm not an expert on this so this is an educated guess: You are calling drop=True and I presume that you want to delete the rows of your dataset that satisfy a condition. That's a problem. If the underlying original data is stored in a dense contiguous array, deleting chunks of it will leave it with "holes". Unless the backend supports sparse implementations, it is likely that it will go for the easiest solution: copy the non-deleted rows in a new array. I don't know the details of you particular problem but most of the time the trick is in not letting the whole data to be loaded. Try to see if instead of loading all the dataset and then performing the filtering/selection, you can do the filtering during the loading. An alternative could use filtering "before" doing the real work. For example, if you have a CSV of >100GB you could write a program X that copies the dataset into a new CSV but doing the filtering. Then, you load the filtered dataset and do the real work in a program Y. I explicitly named X and Y as, in principle, they are 2 different programs using even 2 different technologies. I hope this email can give you hints of how to fix it. In my last project I had a similar problem and I ended up doing the filtering on Python and the "real work" in Julia. Thanks! Martin. On Mon, Jun 06, 2022 at 02:28:41PM -0800, Israel Brewster wrote: I have some large (>100GB) datasets loaded into memory in a two-dimensional (X and Y) NumPy array backed XArray dataset. At one point I want to filter the data using a boolean array created by performing a boolean operation on the dataset that is, I want to filter the dataset for all points with a longitude value greater than, say, 50 and less than 60, just to give an example (hopefully that all makes sense?). Currently I am doing this by creating a boolean array (data[‘latitude’]>50, for example), and then applying that boolean array to the dataset using .where(), with drop=True. This appears to work, but has two issues: 1) It’s slow. On my large datasets, applying where can take several minutes (vs. just seconds to use a boolean array to index a similarly sized numpy array) 2) It uses large amounts of memory (which is REALLY a problem when the array is already using 100GB+) What it looks like is that values corresponding to True in the boolean array are copied to a new XArray object, thereby potentially doubling memory usage until it is complete, at which point the original object can be dropped, thereby freeing the memory. Is there any solution for these issues? Some way to do an in-place filtering? --- Israel Brewster Software Engineer Alaska Volcano Observatory Geophysical Institute - UAF 2156 Koyukuk Drive Fairbanks AK 99775-7320 Work: 907-474-5172 cell: 907-328-9145 -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Automatic Gain Control in Python?
Den 2022-05-29 skrev Christian Gollwitzer : > Am 29.05.22 um 00:45 schrieb Stefan Ram: >> "Steve GS" writes: >>> Subject: Automatic Gain Control in Python? >> >>Automatic Gain Control in Python is trivial. You have a list >>of samples and normalize them, i.e., divide by max. Slightly >>simplified >> >> [ s/max( samples )for s in samples ] >> >>(where sample values are normalized to the range -1,+1.) > > No, that's not it. Loudness is perceived in a different way, the crudest > music is incredibly loud, and you might have wondered, how they do that. > > Google for "Loudness war" and "dynamic range compression" if you want to > understand it in detail. > I have no suggestions for solving the problem but it struck me that you might be interested in looking up a standard called EBU R128. Start with youtube and you find lectures/demos. Python connection; There is a Python package called ffmpeg-normalize which contains an implementation of EBU R128. AFAIK it works on files, not streaming audio. /Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: Request for assistance (hopefully not OT)
Try to reinstall python and only python and if you succeeds, then try to reinstall the other tools. For this, use "apt-get" instead of "apt" $ sudo apt-get reinstall python3 When a system is heavily broken, be extra careful and read the output of the programs. If "apt-get" says than in order to reinstall python it will have to remove half of your computer, abort. Better ask than sorry. Best of the lucks. Martin. On Tue, May 17, 2022 at 06:20:39AM -0500, o1bigtenor wrote: Greetings I was having space issues in my /usr directory so I deleted some programs thinking that the space taken was more an issue than having older versions of the program. So one of the programs I deleted (using rm -r) was python3.9. Python3.10 was already installed so I thought (naively!!!) that things should continue working. (Python 3.6, 3.7 and 3.8 were also part of this cleanup.) So now I have problems. Following is the system barf that I get when I run '# apt upgrade'. What can I do to correct this self-inflicted problem? (running on debian testing 5.17 Setting up python2.7-minimal (2.7.18-13.1) ... Could not find platform independent libraries Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] /usr/bin/python2.7: can't open file '/usr/lib/python2.7/py_compile.py': [Errno 2] No such file or directory dpkg: error processing package python2.7-minimal (--configure): installed python2.7-minimal package post-installation script subprocess returned error exit status 2 Setting up python3.9-minimal (3.9.12-1) ... update-binfmts: warning: /usr/share/binfmts/python3.9: no executable /usr/bin/python3.9 found, but continuing anyway as you request /var/lib/dpkg/info/python3.9-minimal.postinst: 51: /usr/bin/python3.9: not found dpkg: error processing package python3.9-minimal (--configure): installed python3.9-minimal package post-installation script subprocess returned error exit status 127 dpkg: dependency problems prevent configuration of python3.9: python3.9 depends on python3.9-minimal (= 3.9.12-1); however: Package python3.9-minimal is not configured yet. dpkg: error processing package python3.9 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of python2.7: python2.7 depends on python2.7-minimal (= 2.7.18-13.1); however: Package python2.7-minimal is not configured yet. dpkg: error processing package python2.7 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of python3.9-dev: python3.9-dev depends on python3.9 (= 3.9.12-1); however: Package python3.9 is not configured yet. dpkg: error processing package python3.9-dev (--configure): dependency problems - leaving unconfigured . . . Errors were encountered while processing: python2.7-minimal python3.9-minimal python3.9 python2.7 python3.9-dev -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Changing calling sequence
You probably want something like overload/multiple dispatch. I quick search
on PyPI yields a 'multipledispatch' package.
I never used, however.
On Wed, May 11, 2022 at 08:36:26AM -0700, Tobiah wrote:
On 5/11/22 06:33, Michael F. Stemper wrote:
I have a function that I use to retrieve daily data from a
home-brew database. Its calling sequence is;
def TempsOneDay( year, month, date ):
After using it (and its friends) for a few years, I've come to
realize that there are times where it would be advantageous to
invoke it with a datetime.date as its single argument.
You could just use all keyword args:
def TempsOneDay(**kwargs):
if 'date' in kwargs:
handle_datetime(kwargs['date'])
elif 'year' in kwargs and 'month' in kwargs and 'day' in kwargs:
handle_args(kwargs['year'], kwargs['month'], kwargs['day'])
else:
raise Exception("Bad keyword args")
TempsOneDay(date=datetime.datetime.now)
TempsOneDay(year=2022, month=11, day=30)
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Accuracy of multiprocessing.Queue.qsize before any Queue.get invocations?
If the queue was not shared to any other process, I would guess that its size is reliable. However, a plain counter could be much simpler/safer. The developer of multiprocessing.Queue, implemented size() thinking in how to share the size and maintain a reasonable consistency between process. He/she probably didn't care how well works in a single-process scenario as this is a very special case. Thanks, Martin. On Thu, May 12, 2022 at 06:07:02PM -0500, Tim Chase wrote: The documentation says[1] Return the approximate size of the queue. Because of multithreading/multiprocessing semantics, this number is not reliable. Are there any circumstances under which it *is* reliable? Most germane, if I've added a bunch of items to the Queue, but not yet launched any processes removing those items from the Queue, does Queue.qsize accurately (and reliably) reflect the number of items in the queue? q = Queue() for fname in os.listdir(): q.put(fname) file_count = q.qsize() # is this reliable? # since this hasn't yet started fiddling with it for _ in range(os.cpu_count()): Process(target=myfunc, args=(q, arg2, arg3)).start() I'm currently tracking the count as I add them to my Queue, file_count = 0 for fname in os.listdir(): q.put(fname) file_count += 1 but if .qsize is reliably accurate before anything has a chance to .get data from it, I'd prefer to tidy the code by removing the redunant counting code if I can. I'm just not sure what circumstances the "this number is not reliable" holds. I get that things might be asynchronously added/removed once processes are running, but is there anything that would cause unreliability *before* other processes/consumers run? Thanks, -tkc [1] https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue.qsize -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: windows 11 what is wrong?
Repair passed but pip3 and pip did not find pip.exe and missing modules. Lars Martin hambro Fra: Lars Martin Hambro Sendt: onsdag 27. april 2022, 21:31 Til: python-list@python.org Emne: windows 11 what is wrong? [cid:image001.png@01D85A7E.07A48030] [cid:image002.png@01D85A7E.107B8820] [cid:image005.png@01D85A7E.2DF045D0] Lars Martin Hambro -- https://mail.python.org/mailman/listinfo/python-list
[issue47188] ncurses: *** buffer overflow detected ***: terminated with -D_FORTIFY_SOURCE=3
Martin Liška added the comment: So the issue is ncurses, closing here. -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue47188> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47188] ncurses: *** buffer overflow detected ***: terminated with -D_FORTIFY_SOURCE=3
New submission from Martin Liška :
Note -D_FORTIFY_SOURCE=3 will come newly with GCC12. So I noticed the following
error:
demo.py:
```python
import curses
curses.initscr()
curses.unget_wch('a')
```
Error message:
*** buffer overflow detected ***: terminated
Backtrace:
#0 __pthread_kill_implementation (threadid=,
signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1 0x77d1e1e3 in __pthread_kill_internal (signo=6, threadid=) at pthread_kill.c:78
#2 0x77cce306 in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#3 0x77cb7813 in __GI_abort () at abort.c:79
#4 0x77d111b7 in __libc_message (action=action@entry=do_abort,
fmt=fmt@entry=0x77e573cf "*** %s ***: terminated\n") at
../sysdeps/posix/libc_fatal.c:155
#5 0x77db630a in __GI___fortify_fail (msg=msg@entry=0x77e57375
"buffer overflow detected") at fortify_fail.c:26
#6 0x77db48b6 in __GI___chk_fail () at chk_fail.c:28
#7 0x77db5be8 in __wcrtomb_chk (s=s@entry=0xaae440 "\376\271\255",
wchar=wchar@entry=97 L'a', ps=ps@entry=0x7fffd4f0, buflen=buflen@entry=1)
at wcrtomb_chk.c:31
#8 0x77a18b31 in wcrtomb (__ps=, __wchar=, __s=, __s=, __wchar=,
__ps=) at /usr/include/bits/wchar2.h:402
#9 unget_wch_sp (sp=0xab0920, wch=97 L'a') at
../ncurses/./widechar/lib_unget_wch.c:89
#10 0x77a18b61 in unget_wch (wch=) at
../ncurses/./widechar/lib_unget_wch.c:113
#11 0x77a55be5 in _curses_unget_wch (module=, ch='a') at
/home/marxin/Programming/cpython/Modules/_cursesmodule.c:4497
#12 0x006f6669 in cfunction_vectorcall_O (func=, args=0x77b355b0,
nargsf=, kwnames=0x0) at Objects/methodobject.c:512
#13 0x0042d0e8 in _PyObject_VectorcallTstate (kwnames=0x0,
nargsf=, args=, callable=, tstate=)
at ./Include/cpython/abstract.h:114
#14 PyObject_Vectorcall (kwnames=0x0, nargsf=, args=, callable=) at ./Include/cpython/abstract.h:123
#15 call_function (kwnames=0x0, oparg=, pp_stack=, bounds=0x7fffd640, tstate=0xa70520) at Python/ceval.c:5379
#16 _PyEval_EvalFrameDefault (tstate=, f=,
throwflag=) at Python/ceval.c:3772
So as seen __wcrtomb_chk is called with buflen == 1 and the function aborts if:
size_t
__wcrtomb_chk (char *s, wchar_t wchar, mbstate_t *ps, size_t buflen)
{
/* We do not have to implement the full wctomb semantics since we
know that S cannot be NULL when we come here. */
if (buflen < MB_CUR_MAX)
__chk_fail ();
return __wcrtomb (s, wchar, ps);
}
Where MB_CUR_MAX == 6.
So the question is if the issue is in libcurses library (that is compiler with
-D_FORTIFY_SOURCE=3), or in Modules/_cursesmodule.c?
------
messages: 416495
nosy: Martin Liška
priority: normal
severity: normal
status: open
title: ncurses: *** buffer overflow detected ***: terminated with
-D_FORTIFY_SOURCE=3
___
Python tracker
<https://bugs.python.org/issue47188>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43818] Email does not apply policy to multipart messages with defects
Martin Dengler added the comment: Possible related to https://github.com/python/cpython/pull/32137 and https://bugs.python.org/issue43323 -- ___ Python tracker <https://bugs.python.org/issue43818> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43818] Email does not apply policy to multipart messages with defects
Change by Martin Dengler : -- nosy: +mdengler ___ Python tracker <https://bugs.python.org/issue43818> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43323] UnicodeEncodeError: surrogates not allowed when parsing invalid charset
Change by Martin Dengler : -- nosy: +mdengler ___ Python tracker <https://bugs.python.org/issue43323> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Could frozendict or frozenmap be of some use for PEP 683 (Immortal objects)?
I perhaps didn't understand the PEP completely but I think that the goal of marking some objects as immortal is to remove the refcount from they. For immutable objects that would make them truly immutable. However I don't think that the immortality could be applied to any immutable object by default. Think in the immutable strings (str). What would happen with a program that does heavy parsing? I imagine that it will generate thousands of little strings. If those are immortal, the program will fill its memory very quickly as the GC will not reclaim their memory. The same could happen with any frozenfoo object. Leaving immortality to only a few objects, like True and None makes more sense as they are few (bound if you want). On Wed, Mar 09, 2022 at 09:16:00PM +0100, Marco Sulla wrote: As title. dict can't be an immortal object, but hashable frozendict and frozenmap can. I think this can increase their usefulness. Another advantage: frozen dataclass will be really immutable if they could use a frozen(dict|map) instead of a dict as __dict__ -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Execute in a multiprocessing child dynamic code loaded by the parent process
Then, you must put the initialization (dynamically loading the modules) into the function executed in the foreign process. You could wrap the payload function into a class instances to achieve this. In the foreign process, you call the instance which first performs the initialization and then executes the payload. That's what I have in mind: loading the modules first, and then unpickle and call the real target function. -- https://mail.python.org/mailman/listinfo/python-list
[issue46913] UBSAN: test_ctypes, test_faulthandler and test_hashlib are failing
Martin Panter added the comment: The ctypes overflow is probably the same as described in Issue 28169 and Issue 15119 -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue46913> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Execute in a multiprocessing child dynamic code loaded by the parent process
I understand that yes, pickle.loads() imports any necessary module but only if they can be find in sys.path (like in any "import" statement). Dynamic code loaded from a plugin (which we presume it is *not* in sys.path) will not be loaded. Quick check. Run in one console the following: import multiprocessing import multiprocessing.reduction import pickle pickle.dumps(multiprocessing.reduction.ForkingPickler) In a separated Python console run the following: import pickle import sys 'multiprocessing' in sys.modules False pickle.loads() 'multiprocessing' in sys.modules True So the last check proves that pickle.loads imports any necessary module. Martin. On Mon, Mar 07, 2022 at 08:28:15AM +, Barry wrote: On 7 Mar 2022, at 02:33, Martin Di Paola wrote: Yes but I think that unpickle (pickle.loads()) does that plus importing any module needed Are you sure that unpickle will import code? I thought it did not do that. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: Execute in a multiprocessing child dynamic code loaded by the parent process
Yeup, that would be my first choice but the catch is that "sayhi" may not be a function of the given module. It could be a static method of some class or any other callable. Ah, fair. Are you able to define it by a "path", where each step in the path is a getattr() call? Yes but I think that unpickle (pickle.loads()) does that plus importing any module needed in the path which it is handy because I can preload the plugins (modules) before the unpickle but the path may contain others more-standard modules as well. Something like "myplugin.re.match". unpickle should import 're' module automatically will it is loading the function "match". Fair. I guess, then, that the best thing to do is to preload the modules, then unpickle. So, basically what you already have, but with more caveats. Yes, this will not be transparent for the user, just trying to minimize the changes needed. And it will require some documentation for those caveats. And tests. Thanks for the brainstorming! Martin. -- https://mail.python.org/mailman/listinfo/python-list
Re: Execute in a multiprocessing child dynamic code loaded by the parent process
I'm not so sure about that. The author of the plugin knows they're writing code that will be dynamically loaded, and can therefore expect the kind of problem they're having. It could be argued that it's their responsibility to ensure that all the needed code is loaded into the subprocess. Yes but I try to always make my libs/programs as much as usable as possible. "Ergonomic" would be the word. In the case of the plugin-engine I'm trying to hide any side-effect or unexpected behaviour of the engine so the developer of the plugin does not have take that into account. I agree that if the developer uses multiprocessing he/she needs to know its implications. But if I can "smooth" any rough corner, I will try to do it. For example, the main project (developed by me) uses threads for concurrency. It would be simpler to load the plugins and instantiate them *once* and ask the plugins developers to take care of any race condition (RC) within their implementation. Because the plugins were instantiated *once*, it is almost guaranteed that the plugins will suffer from race conditions and they will require some sort of locking. This is quite risky: you may forget to protect something and you will end up with a RC and/or you may put the lock in the wrong place and the whole thing will not work concurrently. My decision back then was to instantiate each plugin N+1 times: once in the main thread and then once per worker thread. With this, no single plugin instance will be shared so there is no risk of RC and no need for locking. (Yes, I know, the developer just needs to use a module variable or a class attribute and it will get a RC and these are shared but it is definitely not the default scenario). If sharing is required I provide an object that minimizes the locking needed. It was much complex for me at the design and at the implementation level but I think that it is safer and requires less from the plugin developer. Reference: https://byexamples.github.io/byexample/contrib/concurrency-model -- https://mail.python.org/mailman/listinfo/python-list
Re: Execute in a multiprocessing child dynamic code loaded by the parent process
Try to use `fork` as "start method" (instead of "spawn"). Yes but no. Indeed with `fork` there is no need to pickle anything. In particular the child process will be a copy of the parent so it will have all the modules loaded, including the dynamic ones. Perfect. The problem is that `fork` is the default only in Linux. It works in MacOS but it may lead to crashes if the parent process is multithreaded (and the my is!) and `fork` does not work in Windows. -- https://mail.python.org/mailman/listinfo/python-list
Re: Execute in a multiprocessing child dynamic code loaded by the parent process
The way you've described it, it's a hack. Allow me to slightly redescribe it.
modules = loader()
objs = init(modules)
def invoke(mod, func):
# I'm assuming that the loader is smart enough to not load
# a module that's already loaded. Alternatively, load just the
# module you need, if that's a possibility.
loader()
target = getattr(modules[mod], func)
target()
ch = multiprocessing.Process(target=invoke, args=("some_module", "sayhi"))
ch.start()
Yeup, that would be my first choice but the catch is that "sayhi" may
not be a function of the given module. It could be a static method of
some class or any other callable.
And doing the lookup by hand sounds complex.
The thing is that the use of multiprocessing is not something required by me
(by my plugin-engine), it was a decision of the developer of a particular
plugin so I don't have any control on that.
Using multiprocessing.reduction was a practical decision: if the user
wants to call something non-pickleable, it is not my fault, it is
multiprocessing's fault.
It *would* be my fault if multiprocessing.Process fails only because I'm
loading the code dynamically.
[...] I won't say "the" correct way, as there are other valid
ways, but there's certainly nothing wrong with this idea.
Do you have some in mind? Or may be a project that I could read?
Thanks!
Martin
--
https://mail.python.org/mailman/listinfo/python-list
Execute in a multiprocessing child dynamic code loaded by the parent process
Hi everyone. I implemented time ago a small plugin engine to load code dynamically. So far it worked well but a few days ago an user told me that he wasn't able to run in parallel a piece of code in MacOS. He was using multiprocessing.Process to run the code and in MacOS, the default start method for such process is using "spawn". My understanding is that Python spawns an independent Python server (the child) which receives what to execute (the target function) from the parent process. In pseudo code this would be like: modules = loader() # load the plugins (Python modules at the end) objs = init(modules) # initialize the plugins # One of the plugins wants to execute part of its code in parallel # In MacOS this fails ch = multiprocessing.Process(target=objs[0].sayhi) ch.start() The code fails with "ModuleNotFoundError: No module named 'foo'" (where 'foo' is the name of the loaded plugin). This is because the parent program sends to the serve (the child) what needs to execute (objs[0].sayhi) using pickle as the serialization mechanism. Because Python does not really serialize code but only enough information to reload it, the serialization of "objs[0].sayhi" just points to its module, "foo". Module which it cannot be imported by the child process. So the question is, what would be the alternatives and workarounds? I came with a hack: use a trampoline() function to load the plugins in the child before executing the target function. In pseudo code it is: modules = loader() # load the plugins (Python modules at the end) objs = init(modules) # initialize the plugins def trampoline(target_str): loader() # load the plugins now that we are in the child process # deserialize the target and call it target = reduction.loads(target_str) target() # Serialize the real target function, but call in the child # trampoline(). Because it can be accessed by the child it will # not fail target_str = reduction.dumps(objs[0].sayhi) ch = multiprocessing.Process(target=trampoline, args=(target_str,)) ch.start() The hack works but is this the correct way to do it? The following gist has the minimal example code that triggers the issue and its workaround: https://gist.github.com/eldipa/d9b02875a13537e72fbce4cdb8e3f282 Thanks! Martin. -- https://mail.python.org/mailman/listinfo/python-list
[issue46880] zipfile library doesn't extract windows zip files properly on linux
Martin Fischer added the comment: Can you attach such a .zip file so that others can reproduce the bug? -- nosy: +push-f ___ Python tracker <https://bugs.python.org/issue46880> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46884] [doc] msilib.rst uses data directive to document modules
Martin Fischer added the comment: ([2] was meant to be an online link: https://docs.python.org/3.9/py-modindex.html#cap-m) -- ___ Python tracker <https://bugs.python.org/issue46884> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46884] [doc] msilib.rst uses data directive to document modules
Change by Martin Fischer : -- keywords: +patch pull_requests: +29744 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31620 ___ Python tracker <https://bugs.python.org/issue46884> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46884] [doc] msilib.rst uses data directive to document modules
New submission from Martin Fischer : As per [1] the py:data directive describes data in a module. It should not be used for submodules, that's what the module directive is for. A side-effect that this is causing is that msilib.schema, msilib.sequence and msilib.text do not show up in the Python Module Index[2] as they should. [1]: https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html [2]: file:///home/martin/repos-contrib/cpython/Doc/build/html/py-modindex.html#cap-m -- assignee: docs@python components: Documentation messages: 414209 nosy: docs@python, push-f priority: normal severity: normal status: open title: [doc] msilib.rst uses data directive to document modules type: enhancement versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46884> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46879] [doc] incorrect sphinx object names
Change by Martin Fischer : -- keywords: +patch pull_requests: +29739 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31615 ___ Python tracker <https://bugs.python.org/issue46879> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46879] [doc] incorrect sphinx object names
New submission from Martin Fischer : API members documented in sphinx have an object name, which allow the documentation to be linked from other projects. Sphinx calculates the object name by prefixing the current module name to the directive argument, e.g: .. module:: foo .. function:: bar.baz becomes foo.bar.baz. Since these anchors aren't displayed in the documentation, some mistakes have crept in, namely the Python stdlib documentation currently contains the objects: * asyncio.asyncio.subprocess.DEVNULL * asyncio.asyncio.subprocess.PIPE * asyncio.asyncio.subprocess.STDOUT * asyncio.asyncio.subprocess.Process * multiprocessing.sharedctypes.multiprocessing.Manager * xml.etree.ElementTree.xml.etree.ElementInclude As can be observed in the URL fragments: https://docs.python.org/3/library/asyncio-subprocess.html#asyncio.asyncio.subprocess.Process https://docs.python.org/3/library/multiprocessing.html#multiprocessing.sharedctypes.multiprocessing.Manager https://docs.python.org/3/library/xml.etree.elementtree.html#xml.etree.ElementTree.xml.etree.ElementInclude.default_loader I have a patch, prepared, I'll send a PR straight away. -- assignee: docs@python components: Documentation messages: 414192 nosy: docs@python, push-f priority: normal severity: normal status: open title: [doc] incorrect sphinx object names versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46879> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46877] [doc] unittest.doModuleCleanups() does not exist
Change by Martin Fischer : -- title: unittest.doModuleCleanups() does not exist -> [doc] unittest.doModuleCleanups() does not exist ___ Python tracker <https://bugs.python.org/issue46877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46877] unittest.doModuleCleanups() does not exist
New submission from Martin Fischer : The unittest documentation[1] describes unittest.doModuleCleanups(). That function however doesn't exist since it's only in the unittest.case module and not re-exported in the unittest module (unlike addModuleCleanup). So I think either the documentation should be corrected or doModuleCleanups should be re-exported in unittest/__init__.py to match the documentation. [1]: https://docs.python.org/3.8/library/unittest.html -- assignee: docs@python components: Documentation messages: 414177 nosy: docs@python, lisroach, michael.foord, push-f priority: normal severity: normal status: open title: unittest.doModuleCleanups() does not exist type: enhancement versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Saving/exporting plots from Jupyter-labs?
Den 2022-02-14 skrev Martin Schöön : > > Now I am trying out Jupyter-labs. I like it. I have two head- > scratchers for now: > > 2) Why is Jupyter-labs hooking up to Google-analytics? Now I can answer this one myself. In a tab I had been working my way through a Holoviews tutorial. The tutorial demonstrated how one can open the online Holoviews documentation inside a notebook cell. This is how google-analytics and twitter got 'involved'. Jupyter-labs was not guilty. /Martin -- https://mail.python.org/mailman/listinfo/python-list
[issue46740] Improve Telnetlib's throughput
Change by Martin Kirchgessner : -- keywords: +patch pull_requests: +29577 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31449 ___ Python tracker <https://bugs.python.org/issue46740> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: library not initialized (pygame)
Could you share the traceback / error that you are seeing?
On Sun, May 02, 2021 at 03:23:21PM -0400, Quentin Bock wrote:
Code:
#imports and variables for game
import pygame
from pygame import mixer
running = True
#initializes pygame
pygame.init()
#creates the pygame window
screen = pygame.display.set_mode((1200, 800))
#Title and Icon of window
pygame.display.set_caption("3.02 Project")
icon = pygame.image.load('3.02 icon.png')
pygame.display.set_icon(icon)
#setting up font
pygame.font.init()
font = pygame.font.Font('C:\Windows\Fonts\OCRAEXT.ttf', 16)
font_x = 10
font_y = 40
items_picked_up = 0
items_left = 3
def main():
global running, event
#Game Loop
while running:
#sets screen color to black
screen.fill((0, 0, 0))
#checks if the user exits the window
for event in pygame.event.get():
if event.type == pygame.QUIT:
running = False
pygame.quit()
def display_instruction(x, y):
instructions = font.render("Each level contains 3 items you
must pick up in each room.", True, (255, 255, 255))
instructions_2 = font.render("When you have picked up 3 items,
you will advance to the next room, there are 3.", True, (255, 255, 255))
instructions_3 = font.render("You will be able to change the
direction you are looking in the room, this allows you to find different
objects.", True, (255, 255, 255))
clear = font.render("Click to clear the text.", True, (255,
255, 255))
screen.blit(instructions, (10, 40))
screen.blit(instructions_2, (10, 60))
screen.blit(instructions_3, (10, 80))
screen.blit(clear, (10, 120))
if event.type == pygame.MOUSEBUTTONDOWN:
if event.type == pygame.MOUSEBUTTONUP:
screen.fill(pygame.Color('black')) # clears the screen text
display_instruction(font_x, font_y)
pygame.display.update()
main()
the error apparently comes from the first instructions variable saying
library not initialized not sure why, its worked before but not now :/
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
That's correct. I tried to be systematic in the analysis so I tested all the possibilities. Your test results were unexpected for `python3 -m venv xxx`. By default, virtual environments exclude the system and user site packages. Including them should require the command-line argument `--system-site-packages`. I'd check sys.path in the environment. Maybe you have PYTHONPATH set. Nope, I checked with "echo $PYTHONPATH" and nothing. I also checked "sys.path" within and without the environment: Inside the environment: ['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/home/user/tmp/xxx/lib/python3.7/site-packages'] Outside the environment: ['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/home/user/.local/lib/python3.7/site-packages', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages'] Indeed the "sys.path" inside the environment does not include system's site-packages. I'll keep looking A virtual environment is configured by a "pyvenv.cfg" file that's either beside the executable or one directory up. Activating an environment is a convenience, not a requirement. Thanks, that makes a little more sense! -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
[issue46756] Incorrect authorization check in urllib.request
Martin Panter added the comment: Maybe the same as Issue 42766? -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue46756> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46738] Allow http.server to emit HTML 5
Change by Martin Panter : -- superseder: -> Generate HTML 5 with SimpleHTTPRequestHandler.list_directory ___ Python tracker <https://bugs.python.org/issue46738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: venv and executing other python programs
If you have activated the venv then any script that uses /usr/bin/env will use executables from the venv bin folder. That's correct. I tried to be systematic in the analysis so I tested all the possibilities. I avoid all these issues by not activating the venv. Python has code to know how to use the venv libraries that are installed in it when invoked. It does not depend on the activate script being run. I had to test this myself because I didn't believe it but you are right. Without having the venv activated, if the shebang explicitly points to the python executable of the venv, the program will have access to the libs installed in the environment. The same if I do: /home/user/venv/bin/python foo.py Thanks for the info! Barry Do you have a particular context where you are having troubles? May be there is something else going on... Thanks, Martin. On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote: Hi, I have recently started using venv for my hobby-programming. There is an annoying problem. Since venv modifies $PATH, python programs that use the "#!/usr/bin/env python" variant of the hashbang often fail since their additional modules aren't install inside in venv. How to people here deal with that? Please note: I'm not interested in discussing whether the env-variant is good or bad. ;-) It's not that *I* use it, but several progs in /usr/bin/. Thanks for your time. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Saving/exporting plots from Jupyter-labs?
Den 2022-02-15 skrev Reto : > On Mon, Feb 14, 2022 at 08:54:01PM +0000, Martin Schöön wrote: >> 1) In notebooks I can save a plot by right-clicking on it and do >> save image as. In Jupyter-lab that does not work and so far I >> have not been able to figure out how to do it. Yes, I have looked >> in the documentation. > > Shift + right click brings up the usual browser menu Thanks, /Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
I did a few experiments in my machine. I created the following foo.py
import pandas
print("foo")
Now "pandas" is installed under Python 3 outside the venv. I can run it
successfully calling "python3 foo.py".
If I add the shebang "#!/usr/bin/env python3" (notice the 3), I can also
run it as "./foo.py".
Calling it as "python foo.py" or using the shebang "#!/usr/bin/env
python" does not work and it makes sense since "pandas" is installed
only for python 3.
New I create a virtual env with "python3 -m venv xxx" and activate it.
Once inside I can run foo.py in 4 different ways:
- python foo.py
- python3 foo.py
- ./foo.py (using the shebang "#!/usr/bin/env python")
- ./foo.py (using the shebang "#!/usr/bin/env python3")
Now all of that was with "pandas" installed outside of venv but not
inside.
I did the same experiments with another library, "cryptonita" which it
is not installed outside but inside and I was able to executed in
4 different ways too (inside the venv of course):
- python foo.py
- python3 foo.py
- ./foo.py (using the shebang "#!/usr/bin/env python")
- ./foo.py (using the shebang "#!/usr/bin/env python3")
Do you have a particular context where you are having troubles? May be
there is something else going on...
Thanks,
Martin.
On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
Hi,
I have recently started using venv for my hobby-programming. There
is an annoying problem. Since venv modifies $PATH, python programs
that use the "#!/usr/bin/env python" variant of the hashbang often
fail since their additional modules aren't install inside in venv.
How to people here deal with that?
Please note: I'm not interested in discussing whether the
env-variant is good or bad. ;-) It's not that *I* use it, but
several progs in /usr/bin/.
Thanks for your time.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Saving/exporting plots from Jupyter-labs?
I have used Jupyter notebooks for some time now. I am not a heavy or advanced user. I find the notebook format a nice way to create python documents. Now I am trying out Jupyter-labs. I like it. I have two head- scratchers for now: 1) In notebooks I can save a plot by right-clicking on it and do save image as. In Jupyter-lab that does not work and so far I have not been able to figure out how to do it. Yes, I have looked in the documentation. 2) Why is Jupyter-labs hooking up to Google-analytics? TIA, /Martin -- https://mail.python.org/mailman/listinfo/python-list
[issue46740] Improve Telnetlib's throughput
New submission from Martin Kirchgessner : While using `telnetlib` I sometimes received unusually "large" messages (around 1Mb) from another process on the same machine, and was surprised `read_until` took more than a second. After instrumenting I discovered such messages were received at roughly 500kbyte/s. I think this low throughput comes from two implementation details: - `Telnet.fill_rawq` is calling `self.sock.recv(50)`, whereas 4096 is now recommended - the `Telnet.process_rawq` method is transferring from raw queue to cooked queue by appending byte per byte. For the latter, transferring by slices looks much faster (I'm measuring at least 5x). I'm preparing a PR. -- components: Library (Lib) messages: 413195 nosy: martin_kirch priority: normal severity: normal status: open title: Improve Telnetlib's throughput type: resource usage versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue46740> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: How do you log in your projects?
? Logs are not intended to be read by end users. Logs are primarily used to understand what the code is doing in a production environment. They could also be used to gather metrics data. Why should you log to give a message instead of simply using a print? You are assuming that logs and prints are different but they aren't. It is the same technology: some string formatted in a particular way sent to some place (console or file generally). But using the logging machinery instead a plain print() you win a few features like thread safety and log levels (think in an user that wants to increase the verbose level of the output) When communicating with an user, the logs that are intended to him/her can be sent to the console (like a print) in addition to a file. For user's perspective, they look just like a print. Why? Traceback is vital to understand what and where the problem is. I think you're confusing logs with messages. The stack trace can be logged (I would say must), but the end user generally sees a vague message with some hints, unless the program is used internally only. Yes, that's exactly why the traceback is hidden by default because the user don't care about it. If the error is due something that the user did wrong, then the message should say that and, if possible, add a suggestion of how to do it. For example "The file 'foo.md' was not found." is quite descriptive. If you add to that message a traceback, that will just clutter the console. Tracebacks and other errors and warnings must be logged in a file. I totally agree with that. Specially true when we are talking with server-like software. Tracebacks can be printed to the user if a more verbose output is enabled. In that case you could even pretty print the traceback with syntax highlighting. I guess that this should be discussed case by case. May be you are thinking more in a case where you have a service running and logging and I am more thinking in a program that a human executes by hand. What info and how is presented to the user changes quite a bit. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: How do you log in your projects?
- On a line per line basis? on a function/method basis? In general I prefer logging line by line instead per function. It is easy to add a bunch of decorators to the functions and get the logs of all the program but I most of the time I end up with very confusing logs. There are exceptions, yes, but I prefer the line by line where the log should explain what is doing the code. - Which kind of variable contents do you write into your logfiles? - How do you decide, which kind of log message goes into which level? - How do you prevent logging cluttering your actual code? These three comes to the same answer: I think on whom is going to read the logs. If the logs are meant to be read by my users I log high level messages, specially before parts that can take a while (like the classic "Loading..."). If I log variables, those must be the ones set by the users so he/she can understand how he/she is controlling the behaviour of the program. For exceptions I print the message but not the traceback. Across the code tag some important functions to put an extra message that will enhance the final message printed to the user. https://github.com/byexamples/byexample/blob/master/byexample/common.py#L192-L238 For example: for example in examples: with enhance_exceptions(example, ...): foo() So if an exception is raised by foo(), enhance_exceptions() will attach to it useful information for the user from the example variable. In the main, then I do the pretty print https://github.com/byexamples/byexample/blob/master/byexample/byexample.py#L17-L22 If the user of the logs is me or any other developer I write more debugging stuff. My approach is to not log anything and when I have to debug something I use a debugger + some prints. When the issue is fixed I review which prints would be super useful and I turn them into logs and the rest is deleted. On Tue, Feb 08, 2022 at 09:40:07PM +0100, Marco Sulla wrote: These are a lot of questions. I hope we're not off topic. I don't know if mine are best practices. I can tell what I try to do. On Tue, 8 Feb 2022 at 15:15, Lars Liedtke wrote: - On a line per line basis? on a function/method basis? I usually log the start and end of functions. I could also log inside a branch or in other parts of the function/method. - Do you use decorators to mark beginnings and ends of methods/functions in log files? No, since I put the function parameters in the first log. But I think that such a decorator it's not bad. - Which kind of variable contents do you write into your logfiles? Of course you shouldn't leak secrets... Well, all the data that is useful to understand what the code is doing. It's better to repeat the essential data to identify a specific call in all the logs of the function, so if it is called simultaneously by more clients you can distinguish them - How do you decide, which kind of log message goes into which level? It depends on the importance, the verbosity and the occurrences of the logs. - How do you prevent logging cluttering your actual code? I have the opposite problem, I should log more. So I can't answer your question. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Waht do you think about my repeated_timer class
In _run I first set the new timer and then I execute the function. So that will go mostly OK. Yes, that's correct however you are not taking into consideration the imprecision of the timers. Timer will call the next _run() after self._interval *plus* some unknown arbitrary time (and extra delay). Let's assume that when you setup an 1 sec Timer but the Timer calls _run() after 1.01 secs due this unknown extra delay. The first time fn() should be called after 1 sec since the begin but it is called after 1.01 secs so the extra delay was of 0.01 sec. The second time fn() should be called after 2 secs since the begin but it is called after 2.02 secs. The second fn() was delayed not by 0.01 but by 0.02 secs. The third fn() will be delayed by 0.03 secs and so on. This arbitrary delay is very small however it will sum up on each iteration and depending of your application can be a serious problem. I wrote a post about this and how to create "constant rate loops" which fixes this problem: https://book-of-gehn.github.io/articles/2019/10/23/Constant-Rate-Loop.html In the post I also describe two solutions (with their trade-offs) for when the target function fn() takes longer than the self._interval time. See if it helps. Thanks, Martin. On Thu, Feb 03, 2022 at 11:41:42PM +0100, Cecil Westerhof via Python-list wrote: Barry writes: On 3 Feb 2022, at 04:45, Cecil Westerhof via Python-list wrote: Have to be careful that timing keeps correct when target takes a 'lot' of time. Something to ponder about, but can wait. You have noticed that your class does call the function at the repeat interval but rather at the repeat interval plus processing time. Nope: def _next(self): self._timer = Timer(self._interval, self._run) self._timer.start() def _run(self): self._next() self._fn() In _run I first set the new timer and then I execute the function. So that will go mostly OK. The way to fix this is to subtract the last processing elapsed time for the next interval. Sort of a software phase locked loop. Just before you call the run function record the time.time() as start_time. Then you can calculate next_interval = max( .001, interval - time.time() - start_time) I use 1ms as the min interval. But I am working on a complete rewrite to create a more efficient class. (This means I have to change also the code that uses it.) There I have to do something like you suggest. (I am already working on it.) Personally I am also of the opinion that the function should finish in less as 10% from the interval. (That was one of my rewrites.) -- Cecil Westerhof Senior Software Engineer LinkedIn: http://www.linkedin.com/in/cecilwesterhof -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
Martin Forster added the comment: The wheel file was corrupt, due to git taking care of "CRLF/LF" conversion. Sorry for the fuss -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
Martin Forster added the comment: i opened a issue at pyodbc as well, https://github.com/mkleehammer/pyodbc/issues/1015 -- ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
Martin Forster added the comment: it tried one built on my own, and the one from https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyodbc -- ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
New submission from Martin Forster : pyth10n 3.10.1, windows x64 install of yodbc-4.0.32-cp310-cp310-win_amd64.whl crashes. pos: 74196 pos: 74226 pos: -3 ERROR: Could not install packages due to an OSError. Traceback (most recent call last): File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\commands\install.py", line 390, in run installed = install_given_reqs( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\req\__init__.py", line 73, in install_given_reqs requirement.install( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\req\req_install.py", line 758, in install install_wheel( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 794, in install_wheel _install_wheel( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 637, in _install_wheel file.save() File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 404, in save with self._zip_file.open(zipinfo) as f: File "C:\Python310\lib\zipfile.py", line 1520, in open fheader = zef_file.read(sizeFileHeader) File "C:\Python310\lib\zipfile.py", line 743, in read self._file.seek(self._pos) OSError: [Errno 22] Invalid argument Please note i added the line 742 with the content: print(str(self) + ' pos: ' + str(self._pos) ) to help to find the root cause. it Looks like that file.tell, returns a negative number, which kill file.seek. -- components: Windows messages: 411750 nosy: Martin-Forster, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl type: crash versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46517] Review exception handling in urllib
Martin Panter added the comment:
The linked code is for urllib.parse.urlencode, looking something like
try:
if len(query) and not isinstance(query[0], tuple):
raise TypeError
except TypeError:
ty, va, tb = sys.exc_info()
raise TypeError("not a valid non-string sequence "
"or mapping object").with_traceback(tb)
I guess it raises twice so that the error message is not duplicated in the
code. The author probably also wants the TypeError initially raised from the
"len(query)" and "query[0]" operations to get the same "not a valid . . ."
message.
Regarding the OSError, originally it was catching socket.error and raising
IOError. I guess someone only wanted the caller to have catch IOError and not
need to import the socket module. Later these exception types became aliases of
each other.
Anyway, the URLopener class is documented as deprecated, so is it really worth
changing anything in that?
--
nosy: +martin.panter
___
Python tracker
<https://bugs.python.org/issue46517>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46375] io.BytesIO does not have peek()
Marcel Martin added the comment: I opened a PR, but I now wonder whether the missing peek() is by design. First, I noticed that instead of using BytesIO directly, I can wrap the instance in an io.BufferedReader, which does have peek(). (It’s just a bit inconvenient.) The second thing is that BytesIO is currently documented to inherit from BufferedIOBase, but if peek() is implemented, one could argue that BytesIO now should inherit from BufferedReader because it then has all the methods. And that seems to great a change from my perspective. I’ll defer to someone more knowledgeable and do not mind at all if this issue is closed without action. -- ___ Python tracker <https://bugs.python.org/issue46375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46375] io.BytesIO does not have peek()
Change by Marcel Martin : -- keywords: +patch pull_requests: +28996 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30808 ___ Python tracker <https://bugs.python.org/issue46375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: sharing data across Examples docstrings
Hello,
I understand that you want to share data across examples (docstrings)
because you are running doctest to validate them (and test).
The doctest implementation evaluates each docstring separately without
sharing the context so the short answer is "no".
This is a limitation of doctest but it is not the only testing engine
that you can use.
You could use "byexample" ( https://byexamples.github.io ) which it
shares the context by default.
byexample has more features and fixes other caveats of doctests, but
don't take me too serious, I have a natural bias because I'm its author.
If you want to go with byexample, you may want to try its "doctest
compatibility mode" first so you don't have to rewrite any test.
( https://byexamples.github.io/byexample/recipes/python-doctest )
Let me know if it is useful for you.
Thanks,
Martin.
On Tue, Jan 11, 2022 at 04:09:57PM -0600, Sebastian Luque wrote:
Hello,
I am searching for a mechanism for sharing data across Examples sections
in docstrings within a class. For instance:
class Foo:
def foo(self):
"""Method foo title
The example generating data below may be much more laborious.
Examples
>>> x = list("abc") # may be very long and tedious to generate
"""
pass
def bar(self):
"""Method bar title
Examples
>>> # do something else with x from foo Example
"""
pass
Thanks,
--
Seb
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
[issue46375] io.BytesIO does not have peek()
New submission from Marcel Martin : It would be great to be able to use peek() on BytesIO objects. I have a function that gets passed a file-like object and uses peek() on it. This works for nearly all types of files relevant in my library, except BytesIO instances (which I use during testing), for which I need to add a small workaround using tell() and seek(). -- components: Library (Lib) messages: 410552 nosy: marcelm priority: normal severity: normal status: open title: io.BytesIO does not have peek() type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue46375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46352] Steps To Do Arlo Setup
Change by Jacob Martin : -- components: Installation files: Arlo camera setup.jpg nosy: jacobmartin717 priority: normal severity: normal status: open title: Steps To Do Arlo Setup type: security versions: Python 3.8 Added file: https://bugs.python.org/file50558/Arlo camera setup.jpg ___ Python tracker <https://bugs.python.org/issue46352> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46319] datetime.utcnow() should return a timezone aware datetime
Martin Panter added the comment: Perhaps this is a duplicate of Issue 12756? -- nosy: +martin.panter superseder: -> datetime.datetime.utcnow should return a UTC timestamp ___ Python tracker <https://bugs.python.org/issue46319> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: Call julia from Python: which package?
I played with Julia a few months ago. I was doing some data-science
stuff with Pandas and the performance was terrible so I decided to give
Julia a try.
My plan was to do the slow part in Julia and call it from Python.
I tried juliacall (if I don't remember wrong) but I couldn't set up it.
It wasn't as smooth as it was advertised (but hey! you may have a better
luck than me).
The other thing that I couldn't figure out is *how the data is shared
between Python and Julia*. Basically numpy/pandas structures cannot be
used by Julia own libraries as they are, they need to be copied at least
and this can be a real performance hit.
I desisted the idea but you may still consider this as an real option.
Just validate how much data you need to share (in my cases where quite
large datasets, hence the issue).
Having said that, is Julia a real option? May be.
In my case the processing that I needed was quite basic and Julia did
a really good job.
But I felt that the library is too fragmented. In Python you can relay
on Pandas/numpy for processing and matplotlib/seaborn for plotting and
you will 99% covered.
In Julia I need DataFrames.jl, Parquet.jl, CategoricalArrays.jl,
StatsBase.jl, Statistics.jl and Peaks.jl
And I'm not including any plotting stuff.
This fragmentation is not just "inconvenient" because you need to
install more packages but it is more difficult to code.
The API is not consistent so you need to be careful and double check
that what you are calling is really doing what you think.
About the performance, Julia is not magic. It depends on how well it was
coded each package.
In my case I had a good experience except with Parquet.jl which it
didn't understand how to handle categories in the dataset and ended up
loading a lot of duplicated strings and blew up the memory a few times.
I suggest you to write down what you need to speed up and see if it is
implemented in Julia (do a proof of concept). Only then consider to do
the switch.
Good luck and share your results!
Martin.
On Fri, Dec 17, 2021 at 07:12:22AM -0800, Dan Stromberg wrote:
On Fri, Dec 17, 2021 at 7:02 AM Albert-Jan Roskam
wrote:
Hi,
I have a Python program that uses Tkinter for its GUI. It's rather slow so
I hope to replace many or all of the non-GUI parts by Julia code. Has
anybody experience with this? Any packages you can recommend? I found three
alternatives:
* https://pyjulia.readthedocs.io/en/latest/usage.html#
* https://pypi.org/project/juliacall/
* https://github.com/JuliaPy/PyCall.jl
Thanks in advance!
I have zero Julia experience.
I thought I would share this though:
https://stromberg.dnsalias.org/~strombrg/speeding-python/
Even if you go the Julia route, it's probably still best to profile your
Python code to identify the slow ("hot") spots, and rewrite only them.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
[issue40059] Provide a toml module in the standard library
Martin Reboredo added the comment: > Not officially, no. But I'm personally not going to bring it forward right > now. If someone else wants to formulate a complete proposal for the SC on > this then they are definitely welcome to! You will need to address where the > code is coming from, why that code should be used, what's the API, etc. > > The only reason the SC is mentioned here is there will be a discussion about > how to maintain the stdlib, but it simply hasn't happened yet. You don't have > to wait for it and asking for a TOML module might actually force the issue. I've been working on a document detailing the API implementation (as a PEP draft) see https://gitlab.com/YakoYakoYokuYoku/pep-toml. Although I've yet to write the code I'll to move forward with it. -- ___ Python tracker <https://bugs.python.org/issue40059> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46062] In IDLE, 'File > Save As' seems not to allow creation of a new folder
New submission from Martin Whitehead : The problem: In both the IDLE shell and the editor, 'File > Save As' seems not to allow creation of a new folder. My system: IDLE v3.8.10 (64 bit) Python 3.8.10 Tk version 8.6.10 Linus Mint 20.2 Uma (64 bit), which is based on Ubuntu 20.04 LTS (Focal Fossa) -- assignee: terry.reedy components: IDLE messages: 408437 nosy: suffolkpunch, terry.reedy priority: normal severity: normal status: open title: In IDLE, 'File > Save As' seems not to allow creation of a new folder versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue46062> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40059] Provide a toml module in the standard library
Martin Reboredo added the comment: A new python-ideas mail thread was created for this, you can check it out at https://mail.python.org/archives/list/python-id...@python.org/thread/IWJ3I32A4TY6CIVQ6ONPEBPWP4TOV2V7/. -- nosy: +YakoYakoYokuYoku ___ Python tracker <https://bugs.python.org/issue40059> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: threading and multiprocessing deadlock
Hi!, in short your code should work.
I think that the join-joined problem is just an interpretation problem.
In pseudo code the background_thread function does:
def background_thread()
# bla
print("join?")
# bla
print("joined")
When running this function in parallel using threads, you will probably
get a few "join?" first before receiving any "joined?". That is because
the functions are running in parallel.
The order "join?" then "joined" is preserved within a thread but not
preserved globally.
Now, I see another issue in the output (and perhaps you was asking about
this one):
join?
join?
myfnc
myfnc
join?
join?
joined.
joined.
So you have 4 "join?" that correspond to the 4 background_thread
function calls in threads but only 2 "myfnc" and 2 "joined".
Could be possible that the output is truncated by accident?
I ran the same program and I got a reasonable output (4 "join?", "myfnc"
and "joined"):
join?
join?
myfnc
join?
myfnc
join?
joined.
myfnc
joined.
joined.
myfnc
joined.
Another issue that I see is that you are not joining the threads that
you spawned (background_thread functions).
I hope that this can guide you to fix or at least narrow the issue.
Thanks,
Martin.
On Mon, Dec 06, 2021 at 12:50:11AM +0100, Johannes Bauer wrote:
Hi there,
I'm a bit confused. In my scenario I a mixing threading with
multiprocessing. Threading by itself would be nice, but for GIL reasons
I need both, unfortunately. I've encountered a weird situation in which
multiprocessing Process()es which are started in a new thread don't
actually start and so they deadlock on join.
I've created a minimal example that demonstrates the issue. I'm running
on x86_64 Linux using Python 3.9.5 (default, May 11 2021, 08:20:37)
([GCC 10.3.0] on linux).
Here's the code:
import time
import multiprocessing
import threading
def myfnc():
print("myfnc")
def run(result_queue, callback):
result = callback()
result_queue.put(result)
def start(fnc):
def background_thread():
queue = multiprocessing.Queue()
proc = multiprocessing.Process(target = run, args = (queue,
fnc))
proc.start()
print("join?")
proc.join()
print("joined.")
result = queue.get()
threading.Thread(target = background_thread).start()
start(myfnc)
start(myfnc)
start(myfnc)
start(myfnc)
while True:
time.sleep(1)
What you'll see is that "join?" and "joined." nondeterministically does
*not* appear in pairs. For example:
join?
join?
myfnc
myfnc
join?
join?
joined.
joined.
What's worse is that when this happens and I Ctrl-C out of Python, the
started Thread is still running in the background:
$ ps ax | grep minimal
370167 pts/0S 0:00 python3 minimal.py
370175 pts/2S+ 0:00 grep minimal
Can someone figure out what is going on there?
Best,
Johannes
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
[Python-announce] [RELEASE] pyspread 2.0.1
pyspread 2.0.1 == This is a bugfix release for pyspread 2.0. Bug fixes: * Font tests removed because of frequent issues with Debian's font aliasing system * Outdated information regarding SVG export removed from documentation About pyspread == Pyspread is a non-traditional spreadsheet that is based on and written in the programming language Python. The goal of pyspread is to be the most pythonic spreadsheet application. Pyspread is free software. It is released under the GPL v3. Project website: https://pyspread.gitlab.io/ Download page: https://pypi.org/project/pyspread/ Signature for tarball: https://gitlab.com/pyspread/downloads/-/raw/master/releases/pyspread-2.0.tar.gz.sig Source code: https://gitlab.com/pyspread/pyspread Dependencies Mandatory: * Python (≥ 3.6) * numpy (>=1.1) * PyQt5 (≥ 5.10, requires PyQt5.Svg) * setuptools (>=40.0) * markdown2 (>= 2.3) Recommended: * matplotlib (>=1.1.1) * pyenchant (>=1.1) * pip (>=18) * python-dateutil (>= 2.7.0) For building the apidocs with Sphinx see apidocs/requirements.txt Enjoy Martin ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Thanks for your definitive answer, this is what I was waiting for. I understand and I totally agree that subclassing is the way to go to make traceback more flexible. Would you mind linking the other issues concerning the general improvement of traceback? -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Irit, would you be able to take a look at the patch? --- I found another application scenario for the patch: In Numpy and Pandas, the assert_* functions in testing have a __tracebackhide__ local that advises pytest to hide the frame. With the patch in place, we could at least not display any locals if __tracebackhide__ is present. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[Python-announce] [RELEASE] pyspread 2.0
pyspread 2.0 This is the first stable release of pyspread that runs under Python 3 (>=3.6). Apart from version changes and release file updates, it is feature identical with version 1.99.8. About pyspread == Pyspread is a non-traditional spreadsheet that is based on and written in the programming language Python. The goal of pyspread is to be the most pythonic spreadsheet application. Pyspread is free software. It is released under the GPL v3. Project website: https://pyspread.gitlab.io/ Download page: https://pypi.org/project/pyspread/ Signature for tarball: https://gitlab.com/pyspread/downloads/-/raw/master/releases/pyspread-2.0.tar.gz.sig Source code: https://gitlab.com/pyspread/pyspread Dependencies Mandatory: * Python (≥ 3.6) * numpy (>=1.1) * PyQt5 (≥ 5.10, requires PyQt5.Svg) * setuptools (>=40.0) * markdown2 (>= 2.3) Recommended: * matplotlib (>=1.1.1) * pyenchant (>=1.1) * pip (>=18) * python-dateutil (>= 2.7.0) For building the apidocs with Sphinx see apidocs/requirements.txt Enjoy Martin ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
Re: Alternatives to Jupyter Notebook
Den 2021-11-15 skrev Loris Bennett : > Martin Schöön writes: > >> Den 2021-10-20 skrev Shaozhong SHI : >>> >>> My Jupyter notebook becomes unresponsive in browsers. >>> >> Odd, I never had any problems like that. I use Firefox on Linux. >> >>> Are there alternatives to read, edit and run Jupyter Notebook? >>> >> I know some people use emacs orgmode. I have never tried it >> myself and do not know how well it works. > > I don't know Jupyter well enough to know whether Org mode is a real > alternative. However, with Org mode you can combine text and code in > multiple languages, run the code within Emacs and then export the > results in various formats such as PDF and HTML. > I realise I was not making myself clear. Orgmode is great. Full stop. What I don't have experience of is combining orgmode with Jupyter. /Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: Alternatives to Jupyter Notebook
Den 2021-10-20 skrev Shaozhong SHI : > > My Jupyter notebook becomes unresponsive in browsers. > Odd, I never had any problems like that. I use Firefox on Linux. > Are there alternatives to read, edit and run Jupyter Notebook? > I know some people use emacs orgmode. I have never tried it myself and do not know how well it works. /Martin -- https://mail.python.org/mailman/listinfo/python-list
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Thanks Joe! > 1. The changes are sufficient to let the user make things work the way it is > requested that they work and anyone who does not start using the new > format_locals parameter will get the old behavior. That was my goal :) > 2. A side comment: I just noticed that the docstring for > FrameSummary.__init__ does not document the filename, lineno, and name > parameters. Perhaps this could be fixed at the same time? I agree and already updated the pull request. > 3. The new parameter format_locals is slightly confusingly named, because it > does not format a single string from all the locals but instead gives a > dictionary of strings with one string for each local. I think format_locals is OK here (local*s*: plural), but I'm open to better suggestions. > 4. I personally think it would be better to include something like the > example value for format_locals as an extra attribute of the traceback module > so everybody doesn't have to write the same function. That said, the > documented example is sufficient. I sort of agree, but I also don't know what such a general-purpose function would entail. Suggestions? > 5. It is not clear to me why this stringify-ing of the locals can't be done > in StackSummary._extract_from_extended_frame_gen. It passes the dictionary > of locals and also the function to transform it to FrameSummary. It would > make more sense to transform it first. I suppose there might be some user > somewhere who is directly calling FrameSummary so the interface needs to stay. I agree! In principle, FrameSummary has been little more than a container without (much) logic of its own. Memory efficiency requires that that FrameSummary does not hold references to the original locals, thats why repr() was used. I think as well that it would have been better to keep FrameSummary as a mere container and do the conversion of the locals elsewhere. But at this point, this shouldn't be changed anymore. > 6. I fantasize that the new format_locals parameter would have access to the > frame object. In order for this to happen, the frame object would have to be > passed to FrameSummary instead of the 3 values (locals, name, filename) that > are extracted from it. I think the current interface of FrameSummary was > designed to interoperate with very old versions of Python. Oh well. Do you have a use case for that? I have no clue what you would do with the frame object at this point. > 7. If anyone wanted to ever refactor FrameSummary (e.g., to enable my fantasy > in point #6 just above), it becomes harder after this. This change has the > effect of exposing details of the interface of FrameSummary to users of > StackSummary.extract and TracebackException. The four parameters that the > new parameter format_locals takes are most of the parameters of FrameSummary. Previously, I totally misread your request to "not just the local variable values, but also the name of the local variable and maybe also the stack frame it is in". This is why I included filename, lineno and name as parameters to format_locals which now seems totally useless to me and I'll remove them (if nobody objects). -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: I improved the example in traceback.rst to illustrate how format_locals works. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Just to avoid misunderstandings: My pull request is not at all about silencing exceptions. It is about customizing the output of the traceback module. (Just like the introduction of capture_locals previously: #22936) (-X capture_locals, on the other hand, is a hypothetical idea that might help with debugging, but we don't have to discuss this now.) My main argument for the proposed change is that traceback is useless in certain situations, because capture_locals is not "robust" (it *might* raise exceptions; I could handle these in the calling code but that would again hide the stack trace that I was about to investigate) and might dump sensitive data like passwords into logfiles (msg237320, msg237323). Nothing is taken away or hidden, but flexibility is added. (The only other option to resolve these issues would be to re-implement much of the current functionality of traceback in a third-party module, which would lead to maintainability problems and fragmentation.) -- nosy: +rbcollins ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: I see two scenarious discussed here: Scenario 1 (Offline / Batch-Mode): A system runs user-supplied jobs that may fail. In the case of an error, the system shouldn't crash but rather store a maximally helpful message and carry on with the next job. Most likely, the relevant information is the situation that lead to the error in the first place, not that repr() has also gone wrong as a result. This could be a a system to automatically test the homework code of your students. Or, like in my case, a framework that runs experiments. You would very likely want a system that does not crash if a __repr__ is wrongly implemented but prints a readable traceback and carries on with the next job. Here, format_locals could be used to fall back to a different representation of an object if repr() fails. This is the use case that my pull request tries to address primarily. Scenario 2 (Online / Write, Test, Repeat): A user frequently rewrites and executes their code until the desired outcome appears. Errors inevitably happen. In this case, a maximally helpful stack trace is needed. Again, the relevant information is the situation that lead to the error in the first place, not that repr() has also gone wrong as a result. Yes, it would be hard for unexperienced users to come up with StackSummary.extract(format_locals=...) by themselves. But, the correct way would be to use a debugger anyway, not some hand-written code to print exceptions on the fly. However, if your students receive a template file where they fill in missing function bodies etc, there might already be a substantial amount of code which they don't understand at first, nor do they need to care. Therefore, a piece of code that formats exceptions nicely would hurt here. I think the most useful change for Scenario 2 (if, for some reason, a proper debugger is not an option) could be to add a command line option (e.g. -X capture_locals) so that a complete stack trace is printed if an exception bubbles up to interpreter level. (Here, it wouldn't hurt to handle exceptions in repr() because the interpreter already stopped exection anyway.) This option would be very easy to teach to inexperienced users. My pull request would provide preparation for this more extensive change. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39228] traceback.FrameSummary does not handle exceptions from `repr()`
Martin added the comment: I submitted a pull request for the related issue43656: https://github.com/python/cpython/pull/29299 It introduces a format_locals parameter that enables the customized formatting of a FrameSummary. This way, a user of traceback could provide a function that handles exceptions within __repr__ if necessary. -- ___ Python tracker <https://bugs.python.org/issue39228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41544] multiprocessing.dummy.Process lacks daemon parameter
Martin added the comment: Could someone have a look at my pull request? I have trouble with the tests. -- ___ Python tracker <https://bugs.python.org/issue41544> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41544] multiprocessing.dummy.Process lacks daemon parameter
Change by Martin : -- nosy: +davin, pitrou type: -> behavior ___ Python tracker <https://bugs.python.org/issue41544> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Could the participants of this issue please have a look at my pull request: https://github.com/python/cpython/pull/29299 What do you like, what don't you like? Does it work for your use case? -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45665] Problems caused by isinstance(list[int], type) returning True
Martin Rueckl added the comment:
Sorry for the noise:
- Literal['a', 'b'],
- TypeVar('T')
Behave like Optional/Union
--
___
Python tracker
<https://bugs.python.org/issue45665>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45665] Problems caused by isinstance(list[int], type) returning True
Martin Rueckl added the comment:
One thing that probably should be considered in this context:
isinstance(arg, type) == issubclass(type(arg), type)
Holds True for arg in (Optional[X], Union[X, Y]). Both sides evaluate to False.
(v3.10.0)
While I still think both sides evaluating to True would be more intuitive, this
supports the proposed change.
Small test snippet:
```
from typing import Dict, List, Set, Tuple, Optional, Union
import pytest
@pytest.mark.parametrize('arg', [
list, List[int], list[int],
dict, Dict[str, int], dict[str, int],
set, Set[int], set[int],
tuple, Tuple[str, int], tuple[str, int],
Optional[int],
Union[int, str]
])
def test_invariant(arg):
same = isinstance(arg, type) == issubclass(type(arg), type)
result = "Check" if same else "Failed"
print(f"\n{result}: Testing: {arg=} with {type(arg)=}: {isinstance(arg,
type)=} <> {issubclass(type(arg), type)=}")
assert same
```
Any other commonly used annotations that could be added to the checklist?
--
nosy: +martinitus
___
Python tracker
<https://bugs.python.org/issue45665>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: walrus with a twist :+
On 28 Oct 2021 at 18:52:26, "Avi Gross" wrote: > > Ages ago, IBM used a different encoding than ASCII called EBCDIC (Extended > Binary Coded Decimal Interchange Code ) which let them use all 8 bits and > thus add additional symbols. =B1 =A6 =AC IBM started using EBCDIC with System 360 and it is still used on mainframes. -- https://mail.python.org/mailman/listinfo/python-list
[issue23597] Allow easy display of local variables in log messages?
Martin added the comment:
In this pull request[1] I introduced a `format_locals` which can be provided to
customize the serialization of the local variables of each frame. This could be
used to filter out variables with certain names, for example. In this context,
you could use a certain naming convention for variables with sensitive
information and filter them out:
def _format_locals(filename, lineno, name, locals):
return {k: repr(v) for k,v in locals.items() if not
k.endsswith("_sensitive")}
traceback.TracebackException.from_exception(e, capture_locals=True,
format_locals=format_locals).format()
(This should be exactly what Robert was suggesting.)
[1] https://github.com/python/cpython/pull/29299
--
keywords: +patch
message_count: 3.0 -> 4.0
nosy: +moi90
nosy_count: 4.0 -> 5.0
pull_requests: +27572
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/29299
___
Python tracker
<https://bugs.python.org/issue23597>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Change by Martin : -- pull_requests: +27563 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/29299 ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Once again a very good summary, thanks Joe! > it would be quite useful if this function parameter was given not just the > local variable values, but also the name of the local variable and maybe also > the stack frame it is in So this would be something like `format_locals(filename, lineno, name, locals) -> dict[str,str]` that could be used inside FrameSummary.__init__. I like that! I wouldn't bother with detecting un-repr-able objects in Python itself, but if the above `format_locals` was implemented, you could easily prepare such a formatter for your students that hides `self` et al. and/or catch exceptions during `repr` to your liking. What is needed to get an OK for such a change? -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: The task is to invent names for things
IMHO, I prefer really weird names. For example if I'm not sure how to name a class that I'm coding, I name it like XXXYYY (literally). Really ugly. This is a way to avoid the so called "naming paralysis". Once I finish coding the class I look back and it should be easy to see "what it does" and from there, the correct name. If the "what it does" results in multiple things I refactor it, splitting it into two or more pieces and name each separately. Some people prefer using more generic names like "Manager", "Helper", "Service" but those names are problematic. Yes, they fit in any place but that's the problem. If I'm coding a class and I name it as "FooHelper", I may not realize later that the class is doing too many things (unrelated things), because "it is a helper". The thing gets wrong with the time; I bet that most of us saw a "Helper" class with thousands of lines (~5000 lines was my record) that just grows over time. On Wed, Oct 27, 2021 at 12:41:56PM +0200, Karsten Hilbert wrote: Am Tue, Oct 26, 2021 at 11:36:33PM + schrieb Stefan Ram: xyzzy = lambda x: 2 * x . Sometimes, this can even lead to "naming paralysis", where one thinks excessively long about a good name. To avoid this naming paralysis, one can start out with a mediocre name. In the course of time, often a better name will come to one's mind. In that situation, is it preferable to choose a nonsensical name over a mediocre one ? Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
[issue45438] inspect not capturing type annotations created by __class_getitem__
Martin Rueckl added the comment: Just my two cents as a new contributor but long time user: - isinstance(list[int], type) returning False seems incredibly un-intuitive to me. I always see generics (e.g. list without type parameter) as higher kinded types, where passing a type argument via [] turns the hkt into a concrete type. - Backporting to 3.9 and 3.10 should be no issue - I am not deep enough in pythons type system to judge whether my PR is consistent/good. I really just tried out what works and "feels consistent". -- ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Irit, I'm unsure about the wording. Something like ":meth:`__repr__` should always succeed, even if errors prevent a full description of the object."? "... even if the object is only partly initialized."? Andrei, I think you can not simply omit the first argument. For virtually all methods, `self` is an important value for debugging. It could be omitted for __init__, but that would break consistency. Also, __init__ could call other methods that help initialize the object and therefore also deal with partly initialized objects. I really think one important part of the solution to this problem is making people aware, that under some rare conditions, repr might receive a partly initialized object. I'm also still convinced that the other part of the solution is a way to customize the behavior of StackSummary.extract and friends. Context: I have written a tool (https://github.com/moi90/experitur) to run batches of (machine learning) experiments (so it is not interactive). In case of an error, the traceback is dumped into a file, together with the respective local variables (using `traceback.TracebackException.from_exception(...).format()`). I want this to succeed *in any case*, even if I was too ignorant to implement correct `__repr__`s in my experiment code (which is mostly on-time-use, so why should I care). In the end, this whole problem only affects users of `capture_locals=True`, so we could just change the semantics of this parameter a bit: False: do nothing, True: use repr (as before), : use this callable to convert the value to a string. This way, we could avoid adding an additional parameter to many of the methods in traceback AND give users an easy way to customize exception formatting for easy debugging. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45438] inspect not capturing type annotations created by __class_getitem__
Martin Rueckl added the comment: I just created a PR and signed the contributor agreement. Waiting for it to update :-) Comments welcome! -- ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45438] inspect not capturing type annotations created by __class_getitem__
Change by Martin Rueckl : -- keywords: +patch pull_requests: +27476 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29212 ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45438] inspect not capturing type annotations created by __class_getitem__
Martin Rueckl added the comment: Can confirm for 3.9.7 as well. -- nosy: +martinitus ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: > Can we determine if they came from an initialized object or from object in > the middle of initialization? That would be very nice because inside __init__ is the only place where we have to deal with partly initialized objects. But I think Python does not provide a way to detect this state. Although I would very much like having FrameSummary robust to any kind error, I acknowledge that this might not be possible. It might be frustrating for beginners, but I think the only way to "fix" this, is by having people implement their `repr`s correctly. The documentation currently says[1]: > This is typically used for debugging, so it is important that the > representation is information-rich and unambiguous. It should be added that __repr__ might be used to display partly initialized objects during debugging and therefore should deal with these gracefully. [1] https://docs.python.org/3/reference/datamodel.html#object.__repr__ -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45585] Gzipping subprocess output produces invalid .gz file
Martin Panter added the comment: The subprocess module only uses the file object to get a file handle by calling the "fileno" method. See Issue 19992 about documenting this. For Python to compress the output of the child process, you would need a pipe. Gzip file objects provide the "fileno" method, but it just returns the underlying file descriptor. Data written to that file descriptor would normally already be compressed by Python and goes straight to the OS. There is also Issue 24358 opened about whether "fileno" should be implemented. -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue45585> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
