Re: PEP668 / pipx and "--editable" installs
Am Sat, Sep 16, 2023 at 02:17:19PM +1200 schrieb Rimu Atkinson via Python-list: > Everyone uses virtual environments. Umm, like, no. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
type annotation vs working code
A type annotation isn't supposed to change what code does,
or so I thought:
#
class Borg:
_instances:dict = {}
def __new__(cls, *args, **kargs):
# look up subclass instance cache
if Borg._instances.get(cls) is None:
Borg._instances[cls] = object.__new__(cls)
return Borg._instances[cls]
class WorkingSingleton(Borg):
def __init__(self):
print(self.__class__.__name__, ':')
try:
self.already_initialized
print('already initialized')
return
except AttributeError:
print('initializing')
self.already_initialized = True
self.special_value = 42
class FailingSingleton(Borg):
def __init__(self):
print(self.__class__.__name__, ':')
try:
self.already_initialized:bool
print('already initialized')
return
except AttributeError:
print('initializing')
self.already_initialized = True
self.special_value = 42
s = WorkingSingleton()
print(s.special_value)
s = FailingSingleton()
print(s.special_value)
#
Notice how Working* and Failing differ in the type annotation
of self.already_initialized only.
Output:
WorkingSingleton :
initializing
42
FailingSingleton :
already initialized <==
Huh ?
Traceback (most recent call last):
File
"/home/ncq/Projekte/gm/git/gnumed/gnumed/client/testing/test-singleton.py",
line 48, in
print(s.special_value)
^^^
AttributeError: 'FailingSingleton' object has no attribute
'special_value'
Where's the error in my thinking (or code) ?
Thanks,
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: type annotation vs working code
Am Sun, Oct 01, 2023 at 09:04:05AM +1300 schrieb dn via Python-list:
> >class WorkingSingleton(Borg):
> >
> > def __init__(self):
> > print(self.__class__.__name__, ':')
> > try:
> > self.already_initialized
> > print('already initialized')
> > return
> >
> > except AttributeError:
> > print('initializing')
> >
> > self.already_initialized = True
> > self.special_value = 42
> >Where's the error in my thinking (or code) ?
>
> What is your thinking?
> Specifically, what is the purpose of testing self.already_initialized?
The purpose is to check whether the singleton class has been
... initialized :-)
The line
self.already_initialized = True
is misleading as to the fact that it doesn't matter at all
what self.already_initialized is set to, as long as is
exists for the next time around.
> Isn't it generally regarded as 'best practice' to declare (and define a value
> for) all
> attributes in __init__()? (or equivalent) In which case, it will (presumably)
> be defined
> as False; and the try-except reworded to an if-else.
I fail to see how that can differentiate between first-call
and subsequent call.
> Alternately, how about using hasattr()? eg
>
> if hasattr( self.already_initialized, 'attribute_name' ):
That does work. I am using that idiom in other children of
Borg. But that's besides the point. I was wondering why it
does not work the same way with and without the type
annotation.
> try:
> self.already_initialized
>
> line is flagged by the assorted linters, etc, in my PyCharm as:
>
> Statement seems to have no effect.
Well, the linter simply cannot see the purpose, which is
test-of-existence.
> Question: is it a legal expression (without the typing)?
It borders on the illegal, I suppose, as the self-
introspection capabilities of the language are being
leveraged to achieve a legal purpose.
Which seems akin constructs for generating compatibility
between versions.
It seems the answer is being pointed to in Matts response.
It just mightily surprised me.
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: type annotation vs working code
Sorry for having conflated the core of the matter with all
the Borg shenanigans, that's where I found the problem in my
real code, so there :-)
Consider this:
#
class Surprise:
def __init__(self, with_type_annotation=False):
if with_type_annotation:
try:
self.does_not_exist:bool
print('does_not_exist does exist')
except AttributeError:
print('does_not_exist does not exist')
return
try:
self.does_not_exist
print('does_not_exist does exist')
except AttributeError:
print('does_not_exist does not exist')
Surprise(with_type_annotation = False)
Surprise(with_type_annotation = True)
#
Is this how it is supposed to be ?
> ...and so we're addressing the important question: the try-test is for
> existence, cf for
> some value.
>
> This can also be achieved by using the attribute in a legal expression, eg:
...
> Might this remove the confusion (ref: @Mats):
>
> self.already_initialized:bool == True
Not for me as that would _create_ already_initialized on the
instance. It would not allow me to test for it.
> >Which seems akin constructs for generating compatibility
> >between versions.
>
> versions of ?
Of the language. Sometimes one tests for existence of a given
class in a module and defines said class oneself if it does
not exist. But that's leading astray.
> What is the intent: a class where each instance is aware of every other
> instance - yet
> the word "Singleton" implies there's only one (cf a dict full of ...)?
The latter.
Regards,
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: type annotation vs working code
Am Wed, Oct 04, 2023 at 05:25:04PM +1300 schrieb dn via Python-list:
> The first question when dealing with the Singleton Pattern is what to do when
> more than
> one instantiation is attempted:
>
> - silently return the first instance
This, in my case.
> and so, returning to the matter of 'readability':
>
> - the name "Borg" de-railed comprehension
>
> - _instances:dict = {} implied the tracking of more than one
Child classes, yes, each being a Singleton.
> or a Singleton() class defined, which is then sub-classed, ie
>
> class Something( Singleton ):
Could have been but the legacy codebase came with Borg ...
> - from there, plenty of 'templates' exist for Singletons,
... which was taken from the Web ages ago.
> - this article (https://python-patterns.guide/gang-of-four/singleton/)
Reading.
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Any possible type alias that can also set a default value for a function arg?
> > or something like that. Basically, any way to avoid writing `= None` over > > and over again. > > Fundamentally no, at least not without some shenanigans. Type hints do > not affect the regular running of the code, Except when they do ;-) ... depending on what counts as (valid) code ... In Python a distinction can be made between "runnable" and "valid" :-D Karsten -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Re: Any possible type alias that can also set a default value for a function arg?
> > > Fundamentally no, at least not without some shenanigans. Type hints do
> > > not affect the regular running of the code,
> >
> > Except when they do ;-)
> >
> > ... depending on what counts as (valid) code ...
> >
> > In Python a distinction can be made between "runnable" and "valid" :-D
> >
>
> Can you give a counter-example?
As per my recent foray into abusing existence-checking for Singleton assurance
along such lines as
>>> try: self.initialized
>>> except AttributeError: print('first instantiation'); self.initialized = True
and then changing that to
>>> try: self.initialized:bool
Karsten
--
https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Re: Re: Any possible type alias that can also set a default value for a function arg?
> > As per my recent foray into abusing existence-checking for Singleton
> > assurance
> > along such lines as
> >
> > >>> try: self.initialized
> > >>> except AttributeError: print('first instantiation'); self.initialized =
> > >>> True
> >
> > and then changing that to
> >
> > >>> try: self.initialized:bool
>
> But that's not equivalent code.
I learned as much (RHS vs LHS).
But it did not _intuitively_ resonate with the sentiment
"type annotation does not change the running of code".
Karsten
--
https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Thu, Nov 02, 2023 at 09:35:43PM - schrieb Jon Ribbens via Python-list: Regardless of ... > Because pip barely plays well by itself, let alone with other package > managers at the same time. ... being true ... > > I do only install a few things using pip. > > Are they not available in your system's package manager? ... this clearly often answers to "no" for applications of any complexity. Is there a suggested proper path to deal with that (Debian is of interest to me here) ? Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list: > >So they now have only python3 and there is no python executable in > >PATH. > > FWIW, for this you install the little stub package python-is-python3. > Especially if you > want to keep a python2 installation around - "python" will still be python3 > in this > case. Since you seem knowledgeable in this area: Do you know of a resource for learning the *canonical* way of packaging a Python application for installation via apt which - needs some packages available via apt - needs some packages only available via pip - needs some packages newer than what is available via apt ? Thanks, Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Fri, Nov 03, 2023 at 05:26:06PM +0100 schrieb Dieter Maurer: > Karsten Hilbert wrote at 2023-11-3 14:47 +0100: > > ... > >> Are they not available in your system's package manager? > > > >... this clearly often answers to "no" for applications of > >any complexity. > > > >Is there a suggested proper path to deal with that (Debian is > >of interest to me here) ? > > Complex applications may maintain a set of "known workable versions" > associated with the application's releases. > They may describe those "known workable versions" in a `pip` constraint file. > In this case, you can upgrade to a new application release > by using this constraint file. Hello Dieter, do you happen to know where to read up on how to fit a pip constraint file into a Debian package creation workflow ? Thanks, Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Fri, Nov 03, 2023 at 01:53:32PM - schrieb Jon Ribbens via Python-list: > >> Are they not available in your system's package manager? > > > > ... this clearly often answers to "no" for applications of > > any complexity. > > > > Is there a suggested proper path to deal with that (Debian is > > of interest to me here) ? > > Yes, as previously mentioned, use virtual environments. > > These days they don't even need to be "activated". For package 'foo' > for example you could create /usr/local/lib/foo, under which you would > create a virtual environment and install the 'foo' package inside it, > and then you could do: > > ln -s /usr/local/lib/foo/env/bin/foo /usr/local/bin/foo > > and then you could just type 'foo' to run it. This all being nice and well, but: How does one "fill" that venv with packages from pip during apt-get install python3-app-of-interest ? Is the suggested way really to pip-install into this venv during apt-get install ? Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Sun, Nov 05, 2023 at 03:00:41PM + schrieb Chris Green via Python-list: > > * contact every single maintainer of every single one of the packages > > that needs updating and persuade them to update their packages and > > reassure them that you are getting all the other package maintainers > > to update their packages accordingly and that you have a plan and > > that you know what you're doing > > > > ... screen fades to black, title card "3 years later", fade in to ... > > > > * publish your package > > > Surely it's not that bad, the vast bulk of Debian, Ubuntu and other > distributions are installed via systems that sort out dependencies once > given a particular package's requirements. Python is surely not > unique in its dependency requirements. Oh, Debian does just fine in resolving dependencies it knows about within its .deb package universe. The problem arises when there's unpackaged modules required. The only answer seems to be to package such modules oneself. If that's been conquered a venv isn't even needed anymore. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Mon, Nov 06, 2023 at 01:17:11AM - schrieb Jon Ribbens via Python-list: > >> >> Are they not available in your system's package manager? > >> > > >> > ... this clearly often answers to "no" for applications of > >> > any complexity. > >> > > >> > Is there a suggested proper path to deal with that (Debian is > >> > of interest to me here) ? > >> > >> Yes, as previously mentioned, use virtual environments. > >> > > How does one "fill" that venv with packages from pip during > > > > apt-get install python3-app-of-interest > > > > ? > > > > Is the suggested way really to pip-install into this venv > > during apt-get install ? > > I don't know what you mean by that. But if you install the apt packages > and then create your venv with --system-site-packages then I believe > your venv should be able to see the apt packages and import them. The problem is when there's modules not available via apt as packages. In that case on needs to either package them or pip them into the venv suggested above. When they are apt-gettable no venv is needed in the first place. One would just install the application script like any other binary, and which loads apt-installed modules just so. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Mon, Nov 06, 2023 at 08:58:00AM +0100 schrieb Dieter Maurer: > I know that debian packagers create debian packages > from Python distributions not using the approach sketched above > and likely they have their reasons. > > You might want to discuss this on an `apt` related mailing list. Yeah, I guess. I know all this stuff about python modules being packaged for Debian and how apt handles dependencies etc etc. I had just hoped someone here might have a handy pointer for how to deal with modules having to be installed from pip for use with an apt-installed python-based application. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: pip/pip3 confusion and keeping up to date
Am Mon, Nov 06, 2023 at 02:43:47PM -0700 schrieb Mats Wichmann via Python-list: > >I had just hoped someone here might have a handy pointer for > >how to deal with modules having to be installed from pip for > >use with an apt-installed python-based application. > > That just shouldn't happen - such packages are supposed to be > dependency-complete within > the packaging universe in question. Yep, that's the preferable ideal world. Which doesn't happen (but that's not the fault of anyone around here, no harm intended). .From all the posts I gather the answer to my question is "simply": unpackaged-but-needed modules need to be packaged. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: pip/pip3 confusion and keeping up to date
> > .From all the posts I gather the answer to my question is > > "simply": unpackaged-but-needed modules need to be packaged. > > I think there is one aspect that isn't getting consideration here. And > that is whether or not you want these packages installed in the default > system Python install. You might not. Indeed, which is why all the fuzz about how to fill-in a venv from pip while installing with apt :-) With "properly" packaged modules one wouldn't risk (that much) system breakage, at any rate. > Maybe you want to get the latest > possible version of super-dooper-gui-helper, but one of its dependencies > doesn't play well with the system Python libraries. Or ... but you get > the point. There are probably many cases where you want *not* to > install into the system Python world. So you would need to come up with > an APT-based installer that doesn't do that. > > Obviously it's not unthinkable; Certainly not, it's just that I had hoped someone goes: look here and all of this ... > it is just one more thing to figure out. ... has been thought through before. Thanks, Karsten -- https://mail.python.org/mailman/listinfo/python-list
mypy question
Hi all, I am not sure why mypy thinks this gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected "List[Dict[str, Union[str, List[Any], Dict[str, Any" [arg-type] rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True) ^~~ should be flagged. The intent is for "queries" to be a list of dicts with keys of str and values of str OR list of anything OR dict with keys of str and values of anything I'd have thunk list[dict[str,str]] matches that ? This is on Python 3.11.2 with mypy 1.0.1 on Debian. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: mypy question
Am Fri, Dec 29, 2023 at 01:15:29PM +0100 schrieb Karsten Hilbert via Python-list: > I am not sure why mypy thinks this > > gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible > type "List[Dict[str, str]]"; expected > "List[Dict[str, Union[str, List[Any], Dict[str, Any" [arg-type] > rows, idx = run_rw_queries(link_obj = conn, queries = > queries, return_data = True) > > ^~~ > > should be flagged. The intent is for "queries" to be > > a list > of dicts > with keys of str > and values of > str OR > list of anything OR > dict with > keys of str > and values of anything > > I'd have thunk list[dict[str,str]] matches that ? > > This is on Python 3.11.2 with mypy 1.0.1 on Debian. For completeness, this was the mypy call signature: mypy --pretty --allow-redefinition --no-strict-optional --ignore-missing-imports --follow-imports silent --show-error-codes --warn-unused-ignores gmPG2.py Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: mypy question
Am Fri, Dec 29, 2023 at 07:49:17AM -0700 schrieb Mats Wichmann via Python-list: > >I am not sure why mypy thinks this > > > >gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible > >type "List[Dict[str, str]]"; expected > >"List[Dict[str, Union[str, List[Any], Dict[str, Any" [arg-type] > > rows, idx = run_rw_queries(link_obj = conn, queries = > > queries, return_data = True) > > > > ^~~ > > > >should be flagged. The intent is for "queries" to be > > > >a list > > of dicts > > with keys of str > > and values of > > str OR > > list of anything OR > > dict with > > keys of str > > and values of anything > > > >I'd have thunk list[dict[str,str]] matches that ? > > Dict[str, str] means the key type and value type should both be strings, Indeed, I know that much, list[dict[str, str]] is what is getting passed in in this particular invocation of run_rw_queries(). For what it's worth here's the signature of that function: def run_rw_queries ( link_obj:_TLnkObj=None, queries:list[dict[str, str | list | dict[str, Any]]]=None, end_tx:bool=False, return_data:bool=None, get_col_idx:bool=False, verbose:bool=False ) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]: Given that I would have thought that passing in list[dict[str, str]] for "queries" ought to be type safe. Mypy indicates otherwise which I am not grokking as to why. > but in your > retelling above you indicate lots of possible value types... actually the > mypy guess > seems to be a pretty good recreation of your psuedo-code description. I agree that mypy's grasp of my intent from queries:list[dict[str, str | list | dict[str, Any]]]=None, into "List[Dict[str, Union[str, List[Any], Dict[str, Any" seems accurate. I just don't understand why list[dict[str, str]] should not pass that construct. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: mypy question
Am Fri, Dec 29, 2023 at 11:04:59AM -0700 schrieb Mats Wichmann via Python-list:
> >For what it's worth here's the signature of that function:
> >
> > def run_rw_queries (
> > link_obj:_TLnkObj=None,
> > queries:list[dict[str, str | list | dict[str, Any]]]=None,
> > end_tx:bool=False,
> > return_data:bool=None,
> > get_col_idx:bool=False,
> > verbose:bool=False
> > ) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:
> >
> >Given that I would have thought that passing in
> >list[dict[str, str]] for "queries" ought to be type safe.
> >Mypy indicates otherwise which I am not grokking as to why.
>
> ah... didn't grok what you were asking, sorry - ignore my attempt then.
Never mind, the attempt to help is appreciated.
> So you are passing something that has been typed more
> narrowly than the function parameter.
That would then sort of skirt on violation of the Liskov
principle, of which I learned while trying to research this
mypy behaviour.
However, I would not think the above to be a narrowing-down
as it just *selects* one of the explicitely "legal" options.
list[dict[str, str | list | dict[str, Any]]]
should AFAICT expand to:
list[dict[str, dict[str, Any]]]
OR
list[dict[str, list]]
OR
list[dict[str, str]]
the last of which should provide coverage of
[{'some key': 'some value'}]
> Can you use a TypeGuard here?
Not from what I understand about them...
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Aw: Re: mypy question
Hi Greg,
> dict[str, str] is not a subtype of dict[str, str | something_else]
> because you can assign a value of type something_else to the latter
> but not the former.
I understand what you are saying but I do not yet understand why this
applies to my situation.
I don't have Python at hand currently, so I'll write untested pseudocode:
def print_greeting(greeting:int|str):
print(greeting)
print_greeting('hello')
The above snippet should be equivalent to my more complicated code over
which mypy complains to the equivalent of
"input" is of type "str"
but expected type "Union[str,int]
I do understand that "str" is formally more narrow than "Union [str,int]" and
the type system has every right to not consider them equivalent.
However, this seems like a very common use case: "allow passing in either str
or int
and have type checking pass either input as valid" -- yet mypy doesn't seem
to share that idea.
Or else there's something I haven't wrapped my head around yet. But what ?
Karsten
--
https://mail.python.org/mailman/listinfo/python-list
Aw: Re: mypy question
Dear Thomas,
thanks for taking the time to look into my issue.
Maybe it helps if I explain what I want (sorry that my web mailer does not
respect
indentation, I will insert dots).
I want a function to run SQL queries:
run_queries(conn, queries):
...for q in queries:
..conn.execute(q)
I now want to add type hints such that my large codebase can
be checked for silly doings. First, queries is to be a list
of the SQL to run:
run_queries(conn, queries:list):
Then, each list entry can be
...a string holding simple, complete SQL (say "SELECT 1")
run_queries(conn, queries:list[str]):
or
...a dict holding the SQL and arguments for parameters
run_queries(conn, queries:list[dict]):
So, taken together:
run_queries(conn, queries:list[str|dict]):
(yes, this is in Python 3.11/3.12)
Now, when it is a list of dicts I want to further constrain the
dicts. Each is to contain the keys "SQL" and "args". So the keys
are of type str. The values for the keys will be of various types,
such that I chose Any as pseudo-type, so that each list entry that
is of type dict should be dict[str, Any], hence:
queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}]
and
run_queries(conn, queries:list[str|dict[str, Any]]):
If I now call this function with a simple SQL query:
SQL_query = 'SELECT 1' # should be type str ?
queries = [SQL_query] # should be type list[str] ?
run_queries(conn, queries = queries)
and run mypy over that (at least inside my complex codebase) I will
get a type mismatch being hinted at.
So far I don't grasp at which point my reasoning above is faulty.
Karsten
--
https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Re: mypy question
> It occurs to me that you could simplify things if you converted those
> plain query strings to dicts:
>
> 'SELECT 1' --> {'SQL': 'SELECT 1'}
Ha, indeed. There's likely not that many "simple string SQL queries"
in that codebase so I shall take it as an opportunity to refactor them.
So, at least that much good has come from the mypy hint ;-)
Karsten
--
https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Re: mypy question
> I'm fairly sure your database queries don't actually give you strings or > dicts, right? You probably get lists (or iterators) of tuples and > somewhere you convert them to the arguments you are feeding to > run_queries(). Ah, no, those queries are enshrined within the middleware as Python strings with placeholdders. When in need of being run they are assembled into a list of dicts-enriched-with-arguments-per-query (and fed to run_queries()). As to what queries *give* me: I have set up psycopg2 to, indeed, hand me a list of dicts (DictRow instances, that is). Those are then used in display rather than being fed to run_queries(). Karsten -- https://mail.python.org/mailman/listinfo/python-list
Re: mypy question
Thanks to all. I ended up using Sequence for the list part and Mapping for the dict part, which does require "import typing" which I would rather have avoided. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: mypy question
Am Fri, Jan 12, 2024 at 02:23:43PM +0100 schrieb Antoon Pardon via Python-list: > > queries:list[dict[str, str | list | dict[str, Any]]]=None, > > > >into > > > > "List[Dict[str, Union[str, List[Any], Dict[str, Any" > > > >seems accurate. I just don't understand why list[dict[str, > >str]] should not pass that construct. > > Sorry for the late reaction ne'er mind ya > I was wondering if > your type hint for queries shouldn't be the following. > > queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str, Ant]]] > > My impression at this moment is that you are write something like: dict[str, > str | int] as > as shorthand for dict[str, str] | dict[str, int]. I do. > But those two are different types. A-ha ! In what way ? Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: mypy question
Am Sat, Jan 13, 2024 at 09:20:00PM +0100 schrieb Karsten Hilbert via
Python-list:
> > I was wondering if
> > your type hint for queries shouldn't be the following.
> >
> > queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str,
> > Ant]]]
Wait, not really. Let me give an example. Here's three times
the same query (as far as PostgreSQL is concerned, after
having been passed through psycopg2):
queries = [
{
'SQL': 'SELECT 1'
},
{
'SQL': 'SELECT %s',
'args': [1]
},
{
'SQL': 'SELECT %(value)s',
'args': {'value': 1}
}
]
The value for key "SQL" will always be str-like.
The value for "args" can be a list or a dict itself.
If "args" is a dict it will be of type [str, Any].
That's what I am trying to tell mypy.
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Extract lines from file, add to new files
> For 30 years I've used a bash script using mailx to send messages to a list > of recipients. They have no salutation to personalize each one. Since I want > to add that personalized salutation I decided to write a python script to > replace the bash script. Why not foxus on just the part you think you are better off using python, namely personalization ? Create personalized files and send them with your trusted mailx solution ? That'll take out wrestling with smptlib et al. Karsten -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Re: Extract lines from file, add to new files
> > Why not foxus on just the part you think you are better off using python, > > namely personalization ? > > > > Create personalized files and send them with your trusted mailx solution ? > > Karsten, > > Too much time. And while mailx accepts the '-a' option for attachments but > has none for individual salutations. It doesn't need to. It just sends the (pre-personalized-by-Python) mail files. Karsten -- https://mail.python.org/mailman/listinfo/python-list
Aw: Re: Re: Re: Extract lines from file, add to new files
> On Tue, 30 Jan 2024, Karsten Hilbert wrote: > > > It doesn't need to. It just sends the (pre-personalized-by-Python) mail > > files. > > Karsten, > > In which case, I might as well have Python format and send the messages. :-) Certainly. But it seems you are wrestling with Python. Might as well reduce the attack surface. Karsten -- https://mail.python.org/mailman/listinfo/python-list
Re: Terminal Emulator (Posting On Python-List Prohibited)
Am Sun, May 19, 2024 at 10:45:09PM +0100 schrieb Barry via Python-list: > > On 18 May 2024, at 16:27, Peter J. Holzer via Python-list > > wrote: > > > > I don't think Linux users have to deal with venvs > > Modern debian (ubuntu) and fedora block users installing using pip. > You must use a venv to pip install packages from pypi now. Which makes one wonder how one is supposed to package Python applications requiring modules not yet packaged by Debian. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
psycopg2: proper positioning of .commit() within try: except: blocks
Dear all,
unto now I had been thinking this is a wise idiom (in code
that needs not care whether it fails to do what it tries to
do^1):
conn = psycopg2.connection(...)
curs = conn.cursor()
try:
curs.execute(SOME_SQL)
except PSYCOPG2-Exception:
some logging being done, and, yes, I
can safely inhibit propagation^1
finally:
conn.commit() # will rollback, if SOME_SQL failed
conn.close()
So today I head to learn that conn.commit() may very well
raise a DB related exception, too:
psycopg2.errors.SerializationFailure: could not serialize access due to
read/write dependencies among transactions
DETAIL: Reason code: Canceled on identification as a pivot, during
commit attempt.
TIP: The transaction might succeed if retried.
Now, what is the proper placement of the .commit() ?
(doing "with ... as conn:" does not free me of committing appropriately)
Should I
try:
curs.execute(SOME_SQL)
conn.commit()
except PSYCOPG2-Exception:
some logging being done, and, yes, I
can safely inhibit propagation^1
finally:
conn.close()# which should .rollback()
automagically in case we had not reached to .commit()
?
Thanks for insights,
Karsten
#---
^1:
This particular code is writing configuration defaults
supplied in-code when no value is yet to be found in the
database. If it fails, no worries, the supplied default
is used by follow-on code and storing it is re-tried next
time around.
#---
Exception details:
Traceback (most recent call last):
File "/usr/share/gnumed/Gnumed/wxpython/gmGuiMain.py", line 3472, in
OnInit
frame = gmTopLevelFrame(None, id = -1, title = _('GNUmed client'),
size = (640, 440))
^
File "/usr/share/gnumed/Gnumed/wxpython/gmGuiMain.py", line 191, in
__init__
self.LayoutMgr = gmHorstSpace.cHorstSpaceLayoutMgr(self, -1)
^^^
File "/usr/share/gnumed/Gnumed/wxpython/gmHorstSpace.py", line 215,
in __init__
self.top_panel = gmTopPanel.cTopPnl(self, -1)
File "/usr/share/gnumed/Gnumed/wxpython/gmTopPanel.py", line 52, in
__init__
wxgTopPnl.wxgTopPnl.__init__(self, *args, **kwargs)
File "/usr/share/gnumed/Gnumed/wxGladeWidgets/wxgTopPnl.py", line 33,
in __init__
self._TCTRL_patient_selector = cActivePatientSelector(self,
wx.ID_ANY, "")
^^^
File "/usr/share/gnumed/Gnumed/wxpython/gmPatSearchWidgets.py", line
1295, in __init__
cfg.get2 (
File "/usr/share/gnumed/Gnumed/pycommon/gmCfg.py", line 248, in get2
self.set (
File "/usr/share/gnumed/Gnumed/pycommon/gmCfg.py", line 367, in set
rw_conn.commit()# will rollback if transaction failed
psycopg2.errors.SerializationFailure: could not serialize access due to
read/write dependencies among transactions
DETAIL: Reason code: Canceled on identification as a pivot, during
commit attempt.
TIP: The transaction might succeed if retried.
2024-08-20 22:17:04 INFO gm.cfg[140274204403392
UpdChkThread-148728] (/usr/share/gnumed/Gnumed/pycommon/gmCfg.py::get2()
#148): creating option [horstspace.update.consider_latest_branch] with default
[True]
2024-08-20 22:17:04 DEBUG gm.db_pool[140274459512896
MainThread]
(/usr/share/gnumed/Gnumed/pycommon/gmConnectionPool.py::exception_is_connection_loss()
#667): interpreting: could not serialize access due to read/write dependencies
among transactions
DETAIL: Reason code: Canceled on identification as a pivot, during
commit attempt.
TIP: The transaction might succeed if retried.
2024-08-20 22:17:04 DEBUG gm.logging[140274459512896
MainThread] (/usr/share/gnumed/Gnumed/pycommon/gmLog2.py::log_stack_trace()
#170): exception: could not serialize access due to read/write dependencies
among transactions
DETAIL: Reason code: Canceled on identification as a pivot, during
commit attempt.
TIP: The transaction might succeed if retried.
2024-08-20 22:17:04 DEBUG gm.logging[140274459512896
MainThread] (/usr/share/gnumed/Gnumed/pycommon/gmLog2.py::log_stack_trace()
#171): type:
2024-08-20 22:17:04 DEBUG gm.logging[140274459512896
MainThread] (/usr/share/gnumed/Gnumed/pycommon
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Sat, Sep 07, 2024 at 09:46:03AM -0700 schrieb Adrian Klaver: > >unto now I had been thinking this is a wise idiom (in code > >that needs not care whether it fails to do what it tries to > >do^1): > > > > conn = psycopg2.connection(...) > > In the above do you have: > > https://www.psycopg.org/docs/extensions.html#psycopg2.extensions.ISOLATION_LEVEL_SERIALIZABLE > > psycopg2.extensions.ISOLATION_LEVEL_SERIALIZABLE I do indeed. > Or is that in some other concurrent transaction? In fact in that codebase all transactions -- running concurrently or not -- are set to SERIALIZABLE. They are not psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT, for that matter. > > curs = conn.cursor() > > try: > > curs.execute(SOME_SQL) > > except PSYCOPG2-Exception: > > some logging being done, and, yes, I > > can safely inhibit propagation^1 > > finally: > > conn.commit() # will rollback, if SOME_SQL failed > > It will if you use with conn:, otherwise it up to you to do the rollback() > > Are you are doing a rollback() in except PSYCOPG2-Exception: ? No I don't but - to my understanding - an ongoing transaction is being closed upon termination of the hosting connection. Unless .commit() is explicitely being issued somewhere in the code that closing of a transaction will amount to a ROLLBACK. In case of SQL having failed within a given transaction a COMMIT will fail-but-rollback, too (explicit ROLLBACK would succeed while a COMMIT would fail and, in-effect, roll back). IOW, when SOME_SQL has failed it won't matter that I close the connection with conn.commit() and it won't matter that conn.commit() runs a COMMIT on the database -- an open transaction having run that failed SQL will still roll back as if ROLLBACK had been issued. Or else my mental model is wrong. https://www.psycopg.org/docs/connection.html#connection.close In the particular case I was writing about the SQL itself succeeded but then the COMMIT failed due to serialization. I was wondering about where to best place any needed conn.commit(). My knee-jerk reaction was to then put it last in the try: block... All this is probably more related to Python than to PostgreSQL. Thanks, Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Sat, Sep 07, 2024 at 01:03:34PM -0700 schrieb Adrian Klaver:
> In the case you show you are doing commit() before the close() so any errors
> in the
> transactions will show up then. My first thought would be to wrap the
> commit() in a
> try/except and deal with error there.
Right, and this was suggested elsewhere ;)
And, yeah, the actual code is much more involved :-D
#
def __safely_close_cursor_and_rollback_close_conn(close_cursor=None,
rollback_tx=None, close_conn=None):
if close_cursor:
try:
close_cursor()
except PG_ERROR_EXCEPTION as pg_exc:
_log.exception('cannot close cursor')
gmConnectionPool.log_pg_exception_details(pg_exc)
if rollback_tx:
try:
# need to rollback so ABORT state isn't retained in
pooled connections
rollback_tx()
except PG_ERROR_EXCEPTION as pg_exc:
_log.exception('cannot rollback transaction')
gmConnectionPool.log_pg_exception_details(pg_exc)
if close_conn:
try:
close_conn()
except PG_ERROR_EXCEPTION as pg_exc:
_log.exception('cannot close connection')
gmConnectionPool.log_pg_exception_details(pg_exc)
#
def run_rw_queries (
link_obj:_TLnkObj=None,
queries:_TQueries=None,
end_tx:bool=False,
return_data:bool=None,
get_col_idx:bool=False,
verbose:bool=False
) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:
"""Convenience function for running read-write queries.
Typically (part of) a transaction.
Args:
link_obj: None, cursor, connection
queries:
* a list of dicts [{'cmd': , 'args': or )
* to be executed as a single transaction
* the last query may usefully return rows, such as:
SELECT currval('some_sequence');
or
INSERT/UPDATE ... RETURNING some_value;
end_tx:
* controls whether the transaction is finalized (eg.
COMMITted/ROLLed BACK) or not, this allows the
call to run_rw_queries() to be part of a framing
transaction
* if link_obj is a *connection* then "end_tx" will
default to False unless it is explicitly set to
True which is taken to mean "yes, you do have full
control over the transaction" in which case the
transaction is properly finalized
* if link_obj is a *cursor* we CANNOT finalize the
transaction because we would need the connection for that
* if link_obj is *None* "end_tx" will, of course, always
be True, because we always have full control over the
connection, not ending the transaction would be pointless
return_data:
* if true, the returned data will include the rows
the last query selected
* if false, it returns None instead
get_col_idx:
* True: the returned tuple will include a dictionary
mapping field names to column positions
* False: the returned tuple includes None instead of a field
mapping dictionary
Returns:
* (None, None) if last query did not return rows
* ("fetchall() result", ) if last query returned any
rows and "return_data" was True
* for *index* see "get_col_idx"
"""
assert queries is not None, ' must not be None'
if link_obj is None:
conn = get_connection(readonly = False)
curs = conn.cursor()
conn_close = conn.close
tx_commit = conn.commit
tx_rollback = conn.rollback
curs_close = curs.close
notices_accessor = conn
else:
conn_close = lambda *x: None
tx_commit = lambda *x: None
tx_rollback = lambda *x: None
curs_close = lambda *x: None
if isinstance(link_obj, dbapi._psycopg.cursor):
curs = link_obj
notices_accessor = curs.connection
elif isinstance(link_obj, dbapi._psycopg.connection):
if end_tx:
tx_commit = link_obj.commit
tx_rollback = link_obj.rollback
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Sat, Sep 07, 2024 at 02:09:28PM -0700 schrieb Adrian Klaver: > >Right, and this was suggested elsewhere ;) > > > >And, yeah, the actual code is much more involved :-D > > > > I see that. > > The question is does the full code you show fail? > > The code sample you show in your original post is doing something very > different then > what you show in your latest post. At this point I do not understand the > exact problem > we are dealing with. We are not dealing with an unsolved problem. I had been asking for advice where to best place that .commit() call in case I am overlooking benefits and drawbacks of choices. The try: do something except: log something finally: .commit() cadence is fairly Pythonic and elegant in that it ensures the the .commit() will always be reached regardless of exceptions being thrown or not and them being handled or not. It is also insufficient because the .commit() itself may elicit exceptions (from the database). So there's choices: Ugly: try: do something except: log something finally: try: .commit() except: log some more Fair but not feeling quite safe: try: do something .commit() except: log something Boring and repetitive and safe(r): try: do something except: log something try: .commit() except: log something I eventually opted for the last version, except for factoring out the second try: except: block. Best, Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Sun, Sep 08, 2024 at 12:48:50PM +1200 schrieb Greg Ewing via Python-list:
> On 8/09/24 9:20 am, Karsten Hilbert wrote:
> > try:
> > do something
> > except:
> > log something
> > finally:
> > .commit()
> >
> >cadence is fairly Pythonic and elegant in that it ensures the
> >the .commit() will always be reached regardless of exceptions
> >being thrown or not and them being handled or not.
>
> That seems wrong to me. I would have thought the commit should only
> be attempted if everything went right.
>
> What if there's a problem in your code that causes a non-SQL-related
> exception when some but not all of the SQL statements in the
> transaction bave been issued? The database doesn't know something
> has gone wrong, so it will happily commit a partially-completed
> transaction and possibly corrupt your data.
A-ha !
try:
run_some_SQL_that_succeeds()
print(no_such_name) #
tongue-in-cheek
1 / 0 # for
good measure
except SOME_DB_ERROR:
print('some DB error, can be ignored for now')
finally:
commit()
which is wrong, given that the failing *Python* statements
may very well belong into the *business level* "transaction"
which a/the database transaction is part of.
See, that's why I was asking in the first place :-)
I was overlooking implications.
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Sun, Sep 08, 2024 at 12:48:50PM +1200 schrieb Greg Ewing via Python-list: > On 8/09/24 9:20 am, Karsten Hilbert wrote: > > try: > > do something > > except: > > log something > > finally: > > .commit() > > > >cadence is fairly Pythonic and elegant in that it ensures the > >the .commit() will always be reached regardless of exceptions > >being thrown or not and them being handled or not. > > That seems wrong to me. I would have thought the commit should only > be attempted if everything went right. It is only attempted when "everything" went right. The fault in my thinking was what the "everything" might encompass. When some SQL fails it won't matter whether I say conn.commit() or conn.rollback() or, in fact, nothing at all -- the (DB !) transaction will be rolled back in any case. However, that reasoning missed this: > What if there's a problem in your code that causes a non-SQL-related > exception when some but not all of the SQL statements in the > transaction bave been [-- even successfully --] issued? Still, in this code pattern: > try: > do something > .commit() > except: > log something it doesn't technically matter whether I say .commit or .rollback here: > .rollback() ... but ... > Doing an explicit rollback ensures that the transaction is always > rolled back if it is interrupted for any reason. explicit is better than implicit ;-) Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Sun, Sep 08, 2024 at 02:58:03PM +0100 schrieb Rob Cliffe via Python-list:
> >Ugly:
> >
> > try:
> > do something
> > except:
> > log something
> > finally:
> > try:
> > .commit()
> > except:
> > log some more
> >
> >Fair but not feeling quite safe:
> >
> > try:
> > do something
> > .commit()
> > except:
> > log something
> >
> >Boring and repetitive and safe(r):
> >
> > try:
> > do something
> > except:
> > log something
> > try:
> > .commit()
> > except:
> > log something
> >
> >I eventually opted for the last version, except for factoring
> >out the second try: except: block.
> Unless I'm missing something, the 1st & 3rd versions always do the commit()
> even after
> the first bit fails, which seems wrong.
Well, it does run a Python function called "commit". That
function will call "COMMIT" on the database. The end result
will be correct (at the SQL level) because the COMMIT will
not effect a durable commit of data when the SQL in "do
something" had failed.
We have, however, elicited that there may well be other
things belonging into the running business level transaction
which may fail and which should lead to data not being
committed despite being technically correct at the SQL level.
> I suggest the 1st version but replacing "finally" by "else". Then the
> try-commit-except
> will not be executed if the "something" fails.
The whole point was to consolidate the commit into one place.
It is unfortunately named, though. It should be called
"close_transaction".
> Perhaps the extra indentation of the second try block is a bit ugly, but it
> is more
> important that it does the right thing.
> If it is convenient (it may not be) to put the whole thing in a function, you
> may feel
> that the follwing is less ugly:
The whole thing does reside inside a function but the exit-early pattern
> try:
> do something
> except:
> log something
> return
> try:
> .commit()
> except:
> log some more
> return
won't help as there's more stuff to be done inside that function.
Thanks,
Karsten
For what it's worth here's the current state of code:
#
def __safely_close_cursor_and_rollback_close_conn(close_cursor=None,
rollback_tx=None, close_conn=None):
if close_cursor:
try:
close_cursor()
except PG_ERROR_EXCEPTION as pg_exc:
_log.exception('cannot close cursor')
gmConnectionPool.log_pg_exception_details(pg_exc)
if rollback_tx:
try:
# need to rollback so ABORT state isn't retained in
pooled connections
rollback_tx()
except PG_ERROR_EXCEPTION as pg_exc:
_log.exception('cannot rollback transaction')
gmConnectionPool.log_pg_exception_details(pg_exc)
if close_conn:
try:
close_conn()
except PG_ERROR_EXCEPTION as pg_exc:
_log.exception('cannot close connection')
gmConnectionPool.log_pg_exception_details(pg_exc)
#
def __log_notices(notices_accessor=None):
for notice in notices_accessor.notices:
_log.debug(notice.replace('\n', '/').replace('\n', '/'))
del notices_accessor.notices[:]
#
def __perhaps_reraise_as_permissions_error(pg_exc, curs):
if pg_exc.pgcode != PG_error_codes.INSUFFICIENT_PRIVILEGE:
return
# privilege problem -- normalize as GNUmed exception
details = 'Query: [%s]' % curs.query.decode(errors =
'replace').strip().strip('\n').strip().strip('\n')
if curs.statusmessage != '':
details = 'Status: %s\n%s' % (
curs.statusmessage.strip().strip('\n').strip().strip('\n'),
details
)
if pg_exc.pgerror is None:
msg = '[%s]' % pg_exc.pgcode
else:
msg = '[%s]: %s' % (pg_exc.pgcode, pg_exc.pgerror)
raise gmExceptions.AccessDenied (
msg,
source = 'PostgreSQL',
code = pg_exc.pgcode,
details = details
)
#
def run_rw_queries (
link_obj:_TLnkObj=None,
queries:_TQueries=None,
end_tx:bool=False,
return_data:bool=None,
get_col_idx:bool=False,
verbo
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Mon, Sep 09, 2024 at 01:48:32PM +1200 schrieb Greg Ewing via Python-list: > That code doesn't inspire much confidence in me. It's far too > convoluted with too much micro-management of exceptions. > > I would much prefer to have just *one* place where exceptions are > caught and logged. I am open to suggestions. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2: proper positioning of .commit() within try: except: blocks
Am Mon, Sep 09, 2024 at 01:48:32PM +1200 schrieb Greg Ewing via Python-list: > That code doesn't inspire much confidence in me. It's far too > convoluted with too much micro-management of exceptions. It is catching two exceptions, re-raising both of them, except for re-raising one of them as another kind of exception. What would you doing differently and how ? > I would much prefer to have just *one* place where exceptions are > caught and logged. There's, of course, a top level handler which logs and user-displays-as-appropriate any exceptions. This is code from a much larger codebase. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2 positioning of .commit() (Posting On Python-List Prohibited)
Am Mon, Sep 09, 2024 at 10:00:11AM - schrieb Jon Ribbens via Python-list: > > The database only needs to commit when it is explicitly told. Anything > > less -- no commit. > > So the Python code is half-way through a transaction when it throws > a (non-database-related) exception and that thread of execution is > aborted. The database connection returns to the pool, and is re-used > by another thread which continues using it to perform a different > sequence of operations ... ending in a COMMIT, which commits > one-and-a-half transactions. Right, but that's true only when writable connections are being pooled, which should be avoidable in many cases. Any pool worth its salt should rollback any potentially pending transactions of a connection when it is given back that pooled connection. Unless explicitely told not to. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2 positioning of .commit() (Posting On Python-List Prohibited)
Am Mon, Sep 09, 2024 at 10:00:11AM - schrieb Jon Ribbens via Python-list: > So the Python code is half-way through a transaction when it throws > a (non-database-related) exception and that thread of execution is > aborted. The database connection returns to the pool, How does it return to the pool ? Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: psycopg2 positioning of .commit() (Posting On Python-List Prohibited)
Am Tue, Sep 10, 2024 at 08:38:30AM - schrieb Jon Ribbens via Python-list: > Ok. So we've moved away from "In any DBMS worth its salt, rollback is > something that happens automatically" Nope. The original post asked something entirely different. > and now you're saying it isn't automatic after all, No again, such shenanigans only start to happen when pooling is brought into the equation. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: Correct syntax for pathological re.search()
Am Tue, Oct 08, 2024 at 08:07:04PM +0100 schrieb MRAB via Python-list:
> >unwanted_tex = '\sout{'
> >if unwanted_tex not in line: do_something_with_libreoffice()
> >
> That should be:
>
> unwanted_tex = r'\sout{'
Hm.
Python 3.11.2 (main, Aug 26 2024, 07:20:54) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> tex = '\sout{'
>>> tex
'\\sout{'
>>>
Am I missing something ?
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: Correct syntax for pathological re.search()
Am Mon, Oct 07, 2024 at 08:35:32AM -0500 schrieb Michael F. Stemper via
Python-list:
> I'm trying to discard lines that include the string "\sout{" (which is TeX,
> for
> those who are curious. I have tried:
> if not re.search("\sout{", line):
> if not re.search("\sout\{", line):
> if not re.search("\\sout{", line):
> if not re.search("\\sout\{", line):
unwanted_tex = '\sout{'
if unwanted_tex not in line: do_something_with_libreoffice()
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: Correct syntax for pathological re.search()
Am Tue, Oct 08, 2024 at 04:59:48PM -0400 schrieb Alan Bawden via Python-list:
> Karsten Hilbert writes:
>
>Python 3.11.2 (main, Aug 26 2024, 07:20:54) [GCC 12.2.0] on linux
>Type "help", "copyright", "credits" or "license" for more
> information.
>>>> tex = '\sout{'
>>>> tex
>'\\sout{'
>>>>
>
>Am I missing something ?
>
> You're missing the warning it generates:
>
> :1: DeprecationWarning: invalid escape sequence '\s'
I knew it'd be good to ask :-D
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: Best Practice Virtual Environment
Am Sat, Oct 05, 2024 at 10:27:33PM +0200 schrieb Ulrich Goebel via Python-list: > Debian (or even Python3 itself) doesn't allow to pip install required > packages system wide, so I have to use virtual environments even there. But > is it right, that I have to do that for every single user? > > Can someone give me a hint to find an howto for that? AFAICT the factual consensus appears to be install modules as packaged by the system you won't need anything else If you do find how to cleanly install non-packaged modules in a system-wide way (even if that means installing every application into its own *system-wide* venv) - do let me know. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: Best Practice Virtual Environment
Am Sun, Oct 06, 2024 at 12:21:09AM +0200 schrieb Karsten Hilbert via Python-list: > Am Sat, Oct 05, 2024 at 10:27:33PM +0200 schrieb Ulrich Goebel via > Python-list: > > > Debian (or even Python3 itself) doesn't allow to pip install required > > packages system wide, so I have to use virtual environments even there. But > > is it right, that I have to do that for every single user? > > > > Can someone give me a hint to find an howto for that? > > If you do find how to cleanly install non-packaged modules > in a system-wide way (even if that means installing every > application into its own *system-wide* venv) - do let me > know. It seems dh-virtualenv is one way to do it. On Debian. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
Re: FileNotFoundError thrown due to file name in file, rather than file itself
Am Tue, Nov 12, 2024 at 09:52:31AM +0100 schrieb Loris Bennett via Python-list:
> Regarding your example above, if 'missingfile.py' contains the following
>
> import configparser
>
> config = configparser.ConfigParser()
>
> try:
> config.read('/foo/bar')
> except FileNotFoundError as e:
> print(f"Error: configuration file {config_file} not found: {e}")
>
> them
>
> python3 missingfile.py
>
> does not produce an any output for me and so does not seem to be a
> reliable way of handling the case where the config file does not exist.
Well:
>>> help(config.read)
Help on method read in module configparser:
read(filenames, encoding=None) method of configparser.ConfigParser
instance
Read and parse a filename or an iterable of filenames.
Files that cannot be opened are silently ignored; this is
designed so that you can specify an iterable of potential
configuration file locations (e.g. current directory, user's
home directory, systemwide directory), and all existing
configuration files in the iterable will be read. A single
filename may also be given.
Return list of successfully read files.
So, the very fact that it does not return any output AND
returns an empty list is the (intended) way of knowing the
error state.
Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list
Re: Using 'with open(...) as ...' together with configparser.ConfigParser.read
Am Thu, Oct 31, 2024 at 07:47:17AM +0100 schrieb Loris Bennett via Python-list: > However I didn't make myself clear: I understand that there are > different functions, depending on whether I have a file name or a > stream. Nevertheless, I just can't think of a practical example where I > might just have *only* a stream, especially one containing my > configuration data. I was just interested to know if anyone can give an > example. Apart from the fact that any data source can be made into a file: one might have a stream of data coming in over, say, http, as in a centralized configuration repository. Karsten -- GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B -- https://mail.python.org/mailman/listinfo/python-list
