Re: PEP668 / pipx and "--editable" installs

2023-09-17 Thread c.buhtz--- via Python-list
Thanks for your reply. btw: I do know what a virtual environment is and
how its works. No need to explain. And again: I do expect "pipx" to use
virtual environments. So this thread is not about pro and cons of using
virtual environments.

On 2023-09-17 11:57 Rimu Atkinson via Python-list
 wrote:
> The first problem can be avoided because virtual environments can use
> a different version of python than the system one. If you need an
> earlier version of python then you can use it instead.

But as I explained I used pipx to install it. So virtual environments
are used in that case.
Maybe I made not the problem clear enough: The problem is that the
installed application ("hyperorg") did not see the system package
"python3-dateutils".

How can I solve this without(!) using another Python version?
The behavior that the pipx-installed application is not able to see
system python packages (installed via "apt install python3-dateutils")
is unexpected to me.

Maybe there is a good reason for that? Explain it please?
Maybe I did something wrong? How can I make it right?
Maybe it is a bug or a use case that is not covered by current version
of pipx or Debian 12?

> The second problem can be avoided because virtual environments exist
> in a part of the file system

Also here I assume I didn't make my problem clear enough.
I tried to install a library (from source) with "pipx". But it did not
work somehow (no clear error message). The global python interpreter
did not see that library.

Why?
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Why doc call `__init__` as a method rather than function?

2023-09-17 Thread Cameron Simpson via Python-list

On 15Sep2023 10:49, scruel tao  wrote:

```python

class A:

...   def __init__(self):
... pass
...
```

On many books and even the official documents, it seems that many authors prefer to call `__init__` 
as a "method" rather than a "function".
The book PYTHON CRASH COURSE  mentioned that "A function that’s part of a class is a 
method.", however, ` A.__init__` tells that `__init__` is a function...


As mentioned, methods in Python _are_ functions for use with a class.


A.__init__



a = A()
a.__init__


>
I wonder how can I call `__init__` as? Consider the output above.
Maybe both are OK?


As you can see, they're both legal expressions.

The thing about `__init__` is that it's usually called automatically 
which you make a new object. Try putting a `print()` call in your 
`__init__` method, then make a new instance of `A`.


The purpose of `__init__` is to initialise the object's attribute/state 
after the basic, empty-ish, object is made.


Like other dunder methods (methods named with double underscores front 
and back) it is called automatically for you. In a subclass the 
`__init__` method calls the subperclass `__init__` and then does 
whatever additional things might be wanted by the subclass.


Let's look at what you used above:

>>> A.__init__


Here's we've just got a reference to the function you supposlied with 
the class definition for class `A`.


This:

>>> a = A()
>>> a.__init__


Here's you've accessed the name `__init__` via an existing instance of 
`A`, your variable `a`. At this point you haven't called it. So you've 
got a callable thing which is a binding of the function to the object 
`a` i.e. when you call it, the "bound method" knows that t is associated 
with `a` and puts that in as the first argument (usually named `self`).


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list


Re: Why doc call `__init__` as a method rather than function?

2023-09-17 Thread Chris Angelico via Python-list
On Mon, 18 Sept 2023 at 13:49, anthony.flury via Python-list
 wrote:
>
>
>
> To me __init__ is a method, but that is implemented internally as
> function associated to a class
>

What is a method, if it is not a function associated with a class?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: PEP668 / pipx and "--editable" installs

2023-09-17 Thread Thomas Passin via Python-list

On 9/16/2023 7:57 PM, Rimu Atkinson via Python-list wrote:



It is nothing bad about using virtual environments but also not about
not using them. In my own work I haven't see a use case where I needed
them. And I expect that some day I'll encounter a use case for it. This
here is not about pro and cons of virtual environments.

You are in a use case where you need them, right now :) When you 
understand the benefits of virtual environments you will understand what 
I meant by that.



Please explain how the two problems I explained are influenced by not
using virtual environments.


The first problem can be avoided because virtual environments can use a 
different version of python than the system one. If you need an earlier 
version of python then you can use it instead.


I have multiple versions of Python on both Windows and Linux machines. 
I don't have to use the system version.  On Windows the "py" launcher 
can launch any Python version on your system.


The second problem can be avoided because virtual environments exist in 
a part of the file system that you have write access to, so you don't 
need to use sudo to install packages. Your main user account does not 
have write access to /usr/bin.


And therefore on Linux pip will install packages as --user even if the 
option isn't specified.  On Windows, one would just use --user routinely.


Also when a virtual environment is activated the path to it's packages 
is a part of that environment so your code will always be able to import 
the packages you want.


Whenever a version of Python is launched its system path is set up so 
that its own packages are used.  The one thing that can cause problems 
is programs in the Python Scripts directory, which may not be on the 
path for a particular version of Python.  A virtual environment will 
take case of that, but I almost never run scripts in that directory so 
it's not an issue for me.


I think there are situations where a venv is useful, but not because of 
the points you have asserted here.  For myself, I find that after a 
while, I tend to forget what I set up the various venvs for, and what 
state they are in.  So I have stopped using them.  I just put up with 
having more packages in /site-packages than a particular applications 
needs. If I want to, say, work on code in a git clone's directory tree, 
I launch Python using a batch file that sets the PYTHONPATH to point there.


It's much easier to understand if you try it for yourself. Google has 
many excellent resources, here is one 
https://www.freecodecamp.org/news/how-to-setup-virtual-environments-in-python/


Best of luck :)

R


--
https://mail.python.org/mailman/listinfo/python-list


Re: Async options for flask app

2023-09-17 Thread Chris Angelico via Python-list
On Mon, 18 Sept 2023 at 13:45, Rimu Atkinson via Python-list
 wrote:
>
> Hi all,
>
> I'm writing a fediverse server app, similar to kbin https://kbin.pub/en
> and lemmy https://join-lemmy.org/. It's like reddit except anyone can
> run a server in the same way email works.

And the part you didn't say: It's like Reddit but without a ridiculous
API fee. Right? :)

> The current fashion is to do this with cooperative multitasking
> (async/await/gevent/etc) to avoid the overhead associated with
> continually context switching threads and processes. Correct me if I'm
> wrong.

I wouldn't say "current fashion", but yes, there is significant
benefit in putting more than one task onto any given thread. (Most
likely, if you want maximum throughput, you'll end up wanting a
two-level multiplexer, with a pool of processes, each one managing a
pool of tasks.)

> How can I do this with Flask? Any async/await tricks? Can I just
> configure gunicorn to use gevent worker threads?
> https://flask.palletsprojects.com/en/2.3.x/deploying/gunicorn/

It's been a while since I did it, but it definitely does work, yes.

> Has anyone tried Quart? https://pypi.org/project/quart/

Not me, so I'll let someone else answer that.

> How well does gevent monkey-patch into a Flask app?
>

Here's one where I've done that: https://github.com/rosuav/mustardmine

However, it's worth noting that (a) the exact details might differ
depending on precisely what you use - this is backed by PostgreSQL and
also uses websockets; and (b) I originally did this quite a while ago,
and things may have gotten easier since then. All I know is, it works
if I do it this way, and I haven't felt the urge to mess with that
part of it. You may notice that my README mentions issues with Python
3.8 from back when that was new :)

But with all of those caveats: Yes, this absolutely does work, and I
make use of the Mustard Mine on a regular basis. It gets updates
occasionally, not often, but I bump it onto newer Pythons every now
and then, so the techniques it uses should still be valid.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Why doc call `__init__` as a method rather than function?

2023-09-17 Thread anthony.flury via Python-list



To me __init__ is a method, but that is implemented internally as 
function associated to a class



When you use A.__init__ on it's own and inspect it, then it will show 
that it is a function object - this is expected. The implementation 
internals of the runtime don't need to have a special implementation for 
a method.


Like all of the other  methods you shouldn't ever need to call 
them directly : these are called dunder methods and represent functions 
and features which are called by other operators.


The only recommended way to call A.__init__ is to create an instance of 
A : obj = A() - the __init__ method gets called automatically with a 
newly created object.


If you did call A.__init__() directly on a an already existing object :

 obj = A()
 A.__init__(obj)

for example - all that would happen is that the object itself would be 
reset : ie the obj's attributes would be reset to whatever the __init__ 
sets them to - if you need to do that it might be better to have a reset 
method.



 Regards,

Tony


-- Original Message --
From: "scruel tao via Python-list" 
To: "python-list@python.org" 
Sent: Friday, 15 Sep, 23 At 11:49
Subject: Why doc call `__init__` as a method rather than function?
```python
class A:
...   def __init__(self):
... pass
...
A.__init__

a = A()
a.__init__
>
```
On many books and even the official documents, it seems that many 
authors prefer to call `__init__` as a "method" rather than a 
"function".
The book PYTHON CRASH COURSE  mentioned that "A function that’s part of 
a class is a method.", however, ` A.__init__` tells that `__init__` is a 
function...

I wonder how can I call `__init__` as? Consider the output above.
Maybe both are OK? If you prefer or think that we must use one of the 
two, please explain the why, I really want to know, thanks!

--
https://mail.python.org/mailman/listinfo/python-list 



-- Anthony Fluryanthony.fl...@btinternet.com
--
https://mail.python.org/mailman/listinfo/python-list


Async options for flask app

2023-09-17 Thread Rimu Atkinson via Python-list

Hi all,

I'm writing a fediverse server app, similar to kbin https://kbin.pub/en 
and lemmy https://join-lemmy.org/. It's like reddit except anyone can 
run a server in the same way email works. This architecture involves a 
lot of inter-server communication, potentially involving thousands of 
different servers.


So, lots of network I/O. Lots of tasks running in parallel to do I/O 
with different servers simultaneously. A fair bit of DB access too.


The current fashion is to do this with cooperative multitasking 
(async/await/gevent/etc) to avoid the overhead associated with 
continually context switching threads and processes. Correct me if I'm 
wrong.


How can I do this with Flask? Any async/await tricks? Can I just 
configure gunicorn to use gevent worker threads? 
https://flask.palletsprojects.com/en/2.3.x/deploying/gunicorn/


Has anyone tried Quart? https://pypi.org/project/quart/

How well does gevent monkey-patch into a Flask app?

A penny for your thoughts

Thanks!

R
--
https://mail.python.org/mailman/listinfo/python-list


Re: PEP668 / pipx and "--editable" installs

2023-09-17 Thread Rimu Atkinson via Python-list




It is nothing bad about using virtual environments but also not about
not using them. In my own work I haven't see a use case where I needed
them. And I expect that some day I'll encounter a use case for it. This
here is not about pro and cons of virtual environments.

You are in a use case where you need them, right now :) When you 
understand the benefits of virtual environments you will understand what 
I meant by that.



Please explain how the two problems I explained are influenced by not
using virtual environments.


The first problem can be avoided because virtual environments can use a 
different version of python than the system one. If you need an earlier 
version of python then you can use it instead.


The second problem can be avoided because virtual environments exist in 
a part of the file system that you have write access to, so you don't 
need to use sudo to install packages. Your main user account does not 
have write access to /usr/bin.


Also when a virtual environment is activated the path to it's packages 
is a part of that environment so your code will always be able to import 
the packages you want.


It's much easier to understand if you try it for yourself. Google has 
many excellent resources, here is one 
https://www.freecodecamp.org/news/how-to-setup-virtual-environments-in-python/


Best of luck :)

R
--
https://mail.python.org/mailman/listinfo/python-list


subprocess: TTYs and preexec_fn

2023-09-17 Thread Antonio Russo via Python-list
Hello!

I'm trying to run a subprocess (actually several), and control
their terminals (not just stdin and stdout).  I have a use case,
but I won't bore everyone with those details.

I'd rather not use pexpect or ptyprocess, because those expose a
very different interface than Popen.  I've mostly acheived my
goals with the following wrapper around subprocess:


> import subprocess, pty, fcntl, termios, os
> TTY = 'termyTTY'
>
> def Popen(cmd, **kwargs):
> tty = kwargs.pop('tty', None)
> if tty is not None:
> if tty is True:
> tty = pty.openpty()
> master_fd, slave_fd = tty
> old_preexec = kwargs.get('preexec_fn')
> def preexec_fn():
> if old_preexec:
> old_preexec()
> fcntl.ioctl(slave_fd, termios.TIOCSCTTY)
> kwargs['preexec_fn'] = preexec_fn
> kwargs['start_new_session'] = True
> for stkind in ('stdin', 'stdout', 'stderr'):
> if kwargs.get(stkind, None) != TTY:
> continue
> kwargs[stkind] = slave_fd
>
> proc = subprocess.Popen(cmd, **kwargs)
> if tty is not None:
> proc.tty = master_fd
> os.close(slave_fd)
> return proc


This adds a tty keyword argument that spawns a new controlling TTY
and switches the new process to it.  It also allows setting stdin/etc.
separately.  You can, for instance, run ssh and send it a password through
the tty channel, and separately receive piped output from the remote ssh
command.

My questions:

1. Is this already available in a simple way somewhere else, preferably with a
Popen-compatible interface?

2. Is this preexec_fn sufficiently simple that it does not run afoul of the
big, scary threading warning of _posixsubprocess.fork_exec ? I.e., I'm
not touching any locks (besides the GIL... but I'm assuming that would be 
properly
handled?).  CAN this be made safe?  I do plan to use threads and locks.

3. Does this seem widely useful enough to be worth trying to get into Python
itself?  It would be nice if this specific behavior, which probably has even
more nuances than I already am aware of, were safely achievable out of the
box.

Best,
Antonio
-- 
https://mail.python.org/mailman/listinfo/python-list


Confusing behavior of PYTHONWARNINGS

2023-09-17 Thread Meowxiik via Python-list

Hello,

For the third time I am trying to work with `PYTHONWARNINGS` filter, and 
for the third time I am having a terrible time. I'd like to seek your 
assistance, I have a few questions:


**Question 1:** Does the environment variable only matter at python 
interpreter startup time? If I set the same variable before python 
start, it seems to work, but it doesn't work trough os.environ. It does 
appear to be hinted at in the docs, but imho not clear enough.


**Question 2:** Why do the following filters not work for filtering 
`urllib3.exceptions.InsecureRequestWarning`


- `PYTHONWARNINGS=ignore:::urllib3.exceptions`

- `PYTHONWARNINGS=ignore:::urllib3`

- `PYTHONWARNINGS=ignore:::urllib3.exceptions`

- `PYTHONWARNINGS=ignore::urllib3.exceptions.InsecureRequestWarning`

None of these filter the warning. The last one has the audacity to 
complain about "invalid module name: 'urllib3.exceptions'" which is very 
confusing to me, given that that specific module is fully importable 
during runtime, which I have tested. The only one I managed to get 
working is `ignore:Unverified`, where "Unverified" is the first word of 
the actual message, however I truly strongly dislike this solution.


Thank you for any help,

Richard

--
https://mail.python.org/mailman/listinfo/python-list


Python Launcher Pops Up When Py-based App Is Running (Mac)

2023-09-17 Thread James Greenham via Python-list
Hello,

On the face of it, the Python-Mac mailing list is largely inactive so I'm 
posting here since it looks like this one is livelier.  

I'm running macOS Mojave with Python 2.7. I have an old Dashboard widget (last 
modified in 2009) that appears to use Python as well. The widget captures the 
full Safari web-page. It works seamlessly on earlier macOS systems but hangs in 
10.14. The widget's core functions constitute code of the script that I'm 
pasting in this message. The problem is that as soon as the execution reaches 
the image processing stage it gets stuck which manifests by Python Launcher 
popping up in Dock and is trying forever until I quit the launcher that 
force-quits the widget showing the error icon.

The issue seems to originate in the doGrab handler (module or whatever the 
correct Python term fits the definition) immediately following the print 
"Processing Image..." line: it's the message the widget displays at getting 
stalled. I( checked that Python 2.7 is installed in the default location at 
/usr/bin. I also have the binaries idle3, idle3.10, pip3, pip3.10 in 
/usr/local/bin, apparently used by another Unix program. 

Is that remedied? The script's environment set to python, i.e. to python 2.7 so 
that shouldn't be the encumbrance. Am I correct?

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# OR:
# coding=utf-8

__version__ = "0.6"

#
# Based on Paul Hammond's webkit2png script - make screenshots of webpages
# http://www.paulhammond.org/webkit2png 
#
# Modified by Paulo Avila for the Page Capture widget
#
# The original source code (v0.5) belongs to Paul Hammond (see notice below).
# Any and all code added after is Copyright (c) 2009 Paulo Avila.
#
# Modification Log by Paul Avila
#
#  2009.04.21 - Changed almost all the output messages in order to behave 
properly with my Page Capture widget
#  2009.04.21 - Added function headers
#  2009.04.21 - Properly flush the output buffers so that messages display 
immediately
#  2009.04.21 - Changed encoding of this file to UTF-8
#  2009.04.25 - Updated how files are named: " > .png"
#  2009.04.25 - Files are now named with the full URL excluding protocol 
(http://) and arguments (...?*)
#



# Copyright (c) 2009 Paul Hammond
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#

import sys
import optparse

try:
  import Foundation
  import WebKit
  import AppKit
  import objc
except ImportError:
  #print "Cannot find pyobjc library files.  Are you sure it is installed?"
  print "Mac OS X 10.5 or higher is required."
  sys.stdout.flush()
  sys.exit() 



class AppDelegate (Foundation.NSObject):
# what happens when the app starts up
def applicationDidFinishLaunching_(self, aNotification):
webview = aNotification.object().windows()[0].contentView()
webview.frameLoadDelegate().getURL(webview)

class WebkitLoad (Foundation.NSObject, WebKit.protocols.WebFrameLoadDelegate):
# what happens if something goes wrong while loading
def webView_didFailLoadWithError_forFrame_(self,webview,error,frame):
#print " ... something went wrong" 
print "Unable to load WebKit."
sys.stdout.flush()
self.getURL(webview)
def 
webView_didFailProvisionalLoadWithError_forFrame_(self,webview,error,frame):
#print " ... something went wrong" 
print "Check URL or Internet connection."
sys.stdout.flush()
self.getURL(webview)



#---
#7 - Generates the filename.

def makeFilename(self,URL,options):
   # make the filename
   if options.filename:
 filename = options.filename
   elif options.md5:
 try:
import md5
 except ImportError:
print "--md5 requires python md5 library"
sys.stdout.flush()
AppKit.NSApplication.sharedApplication().terminate_(None)
 filename = 

Re: Postgresql equivalent of Python's timeit?

2023-09-17 Thread Thomas Passin via Python-list

On 9/17/2023 5:01 AM, Albert-Jan Roskam via Python-list wrote:

On Sep 15, 2023 19:45, "Peter J. Holzer via Python-list"
 wrote:

  On 2023-09-15 17:42:06 +0200, Albert-Jan Roskam via Python-list wrote:
  >    This is more related to Postgresql than to Python, I hope this is
  ok.
  >    I want to measure Postgres queries N times, much like Python timeit
  >    (https://docs.python.org/3/library/timeit.html). I know about
  EXPLAIN
  >    ANALYZE and psql \timing, but there's quite a bit of variation in
  the
  >    times. Is there a timeit-like function in Postgresql?

  Why not simply call it n times from Python?

  (But be aware that calling the same query n times in a row is likely to
  be
  unrealistically fast because most of the data will already be in
  memory.)

=
Thanks, I'll give this a shot. Hopefully the caching is not an issue if I
don't re-use the same database connection.


Here is a stack overflow thread that gives a good rundown of ways to get 
timings from Postgres:


https://stackoverflow.com/questions/9063402/get-execution-time-of-postgresql-query

--
https://mail.python.org/mailman/listinfo/python-list


Re: Postgresql equivalent of Python's timeit?

2023-09-17 Thread Peter J. Holzer via Python-list
On 2023-09-17 11:01:43 +0200, Albert-Jan Roskam via Python-list wrote:
>On Sep 15, 2023 19:45, "Peter J. Holzer via Python-list"
> wrote:
> 
>  On 2023-09-15 17:42:06 +0200, Albert-Jan Roskam via Python-list wrote:
>  >    This is more related to Postgresql than to Python, I hope this is
>  ok.
>  >    I want to measure Postgres queries N times, much like Python timeit
>  >    (https://docs.python.org/3/library/timeit.html). I know about
>  EXPLAIN
>  >    ANALYZE and psql \timing, but there's quite a bit of variation in
>  the
>  >    times. Is there a timeit-like function in Postgresql?
> 
>  Why not simply call it n times from Python?
> 
>  (But be aware that calling the same query n times in a row is likely to
>  be
>  unrealistically fast because most of the data will already be in
>  memory.)
> 
>=
>Thanks, I'll give this a shot. Hopefully the caching is not an issue if I
>don't re-use the same database connection.

There is some per-session caching, but the bulk of it is shared between
sessions or even in the operating system. And you wouldn't want to get
rid of these caches either (which you could do by rebooting or - a bit
faster - restarting postgres and dropping the caches
(/proc/sys/vm/drop_caches on Linux), because that would make the
benchmark unrealistically slow (unless you want to establish some
worst-case baseline). During normal operations some data will be cached,
but probably not all of it and it will change depending on workload and
possibly other factors.

I think Avi's advice to wait for a few minutes between repetitions is
good. Of course that means that you can't just time the whole thing but
have to time each query separately and then compute the average. (On the
bright side that also gives you the opportunity to compute standard
deviation, min, max, quantiles, etc.)

hp

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Postgresql equivalent of Python's timeit?

2023-09-17 Thread Thomas Passin via Python-list

On 9/17/2023 11:48 AM, AVI GROSS via Python-list wrote:

Timing things that are fairly simple is hard enough to do repeatedly, but when 
it involves access to slower media and especially to network connections to 
servers, the number of things that can change are enormous. There are all kinds 
of caching at various levels depending on your hardware and resource contention 
with other programs running here and there as well as on various network-like 
structures and busses or just hard disks. Asking for anything to be repeated 
multiple times in a row as a general rule can make your results seem slower or 
faster depending on too many factors including what else is running on your 
machine.

I am wondering if an approach to running something N times that may average things out a bit is to simply put in a pause. Have your program wait a few minutes between attempts and perhaps even do other things within your loop that make it likely some of the resources you want not to be in a queue have a chance to be flushed as other things take their place. 


One thing I have done for timing queries is to construct a series of 
test queries with the query parameters drawn randomly from a large set 
of values.  The hope is that random combinations will defeat caching and 
provide a reasonably realistic view of the times.



--
https://mail.python.org/mailman/listinfo/python-list


RE: Postgresql equivalent of Python's timeit?

2023-09-17 Thread AVI GROSS via Python-list
Timing things that are fairly simple is hard enough to do repeatedly, but when 
it involves access to slower media and especially to network connections to 
servers, the number of things that can change are enormous. There are all kinds 
of caching at various levels depending on your hardware and resource contention 
with other programs running here and there as well as on various network-like 
structures and busses or just hard disks. Asking for anything to be repeated 
multiple times in a row as a general rule can make your results seem slower or 
faster depending on too many factors including what else is running on your 
machine.

I am wondering if an approach to running something N times that may average 
things out a bit is to simply put in a pause. Have your program wait a few 
minutes between attempts and perhaps even do other things within your loop that 
make it likely some of the resources you want not to be in a queue have a 
chance to be flushed as other things take their place. Obviously, a machine or 
system with lots of resources may take more effort to use enough new data that 
replaces the old.

Good luck. Getting reliable numbers is no easy feat as someone else may have 
trouble duplicating the results with a somewhat different setup.

-Original Message-
From: Python-list  On 
Behalf Of Albert-Jan Roskam via Python-list
Sent: Sunday, September 17, 2023 5:02 AM
To: Peter J. Holzer 
Cc: python-list@python.org
Subject: Re: Postgresql equivalent of Python's timeit?

   On Sep 15, 2023 19:45, "Peter J. Holzer via Python-list"
wrote:

 On 2023-09-15 17:42:06 +0200, Albert-Jan Roskam via Python-list wrote:
 >This is more related to Postgresql than to Python, I hope this is
 ok.
 >I want to measure Postgres queries N times, much like Python timeit
 >(https://docs.python.org/3/library/timeit.html). I know about
 EXPLAIN
 >ANALYZE and psql \timing, but there's quite a bit of variation in
 the
 >times. Is there a timeit-like function in Postgresql?

 Why not simply call it n times from Python?

 (But be aware that calling the same query n times in a row is likely to
 be
 unrealistically fast because most of the data will already be in
 memory.)

   =
   Thanks, I'll give this a shot. Hopefully the caching is not an issue if I
   don't re-use the same database connection.
-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Postgresql equivalent of Python's timeit?

2023-09-17 Thread Albert-Jan Roskam via Python-list
   On Sep 15, 2023 19:45, "Peter J. Holzer via Python-list"
wrote:

 On 2023-09-15 17:42:06 +0200, Albert-Jan Roskam via Python-list wrote:
 >    This is more related to Postgresql than to Python, I hope this is
 ok.
 >    I want to measure Postgres queries N times, much like Python timeit
 >    (https://docs.python.org/3/library/timeit.html). I know about
 EXPLAIN
 >    ANALYZE and psql \timing, but there's quite a bit of variation in
 the
 >    times. Is there a timeit-like function in Postgresql?

 Why not simply call it n times from Python?

 (But be aware that calling the same query n times in a row is likely to
 be
 unrealistically fast because most of the data will already be in
 memory.)

   =
   Thanks, I'll give this a shot. Hopefully the caching is not an issue if I
   don't re-use the same database connection.
-- 
https://mail.python.org/mailman/listinfo/python-list