Re: A Single Instance of an Object?

[Ivan "Rambius" Ivanov via Python-list]

On Mon, Mar 11, 2024 at 5:06 PM dn via Python-list
 wrote:
>
> Good question Rambius!
>
> On 12/03/24 09:53, Ivan "Rambius" Ivanov via Python-list wrote:
> > Hello,
> >
> > I am refactoring some code and I would like to get rid of a global
> > variable. Here is the outline:
> >
> > import subprocess
> >
> > CACHE = {}
>
> First thought: don't reinvent-the-wheel, use lru_cache
> (https://docs.python.org/3/library/functools.html)
>
>
> > The global cache variable made unit testing of the lookup(key) method
> > clumsy, because I have to clean it after each unit test. I refactored
> > it as:
> >
> > class Lookup:
> >  def __init__(self):
> >  self.cache = {}
> >
>
> Change "cache" to be a class-attribute (it is currently an instance.
>
> Then, code AFTER the definition of Lookup can refer to Lookup.cache,
> regardless of instantiation, and code within Lookup can refer to
> self.cache as-is...
>

Thank you for your suggestions. I will research them!

Regards
rambius


-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A Single Instance of an Object?

[Ivan "Rambius" Ivanov via Python-list]

On Mon, Mar 11, 2024 at 5:01 PM Chris Angelico via Python-list
 wrote:
>
> On Tue, 12 Mar 2024 at 07:54, Ivan "Rambius" Ivanov via Python-list
>  wrote:
> > I am refactoring some code and I would like to get rid of a global
> > variable. Here is the outline:
> >
> > ...
> >
> > I have never done that in Python because I deliberately avoided such
> > complicated situations up to now. I know about the Singleton pattern,
> > but I have never implemented it in Python and I don't know if it is
> > Pythonish.
> >
>
> A Singleton is just a global variable. Why do this? Did someone tell
> you "global variables are bad, don't use them"?

I have bad experience with global variables because it is hard to
track what and when modifies them. I don't consider them bad, but if I
can I avoid them.

Regards
rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

A Single Instance of an Object?

[Ivan "Rambius" Ivanov via Python-list]

Hello,

I am refactoring some code and I would like to get rid of a global
variable. Here is the outline:

import subprocess

CACHE = {}

def lookup(key):
Runs the command cmd, parses its output, extract's the key's value,
caches it and returns it. If the key has already been in the cache,
returns its cached value. If the command cmd returns an error, the
value is set to None and cached as None."""

if key in CACHE:
return CACHE[key]

value = None

cmd = f"mycmd {key}"
proc = subprocess(cmd, capture_output=True, text=True, check=False)
if proc.returncode == 0:
value = proc.stdout.strip()
else:
logger.error("cmd returned error")

CACHE[key] = value
return value
...
def main():
while True:
keys = load_keys()
for key in keys:
call_function_that_call_function_that_calls_lookup(key)


The global cache variable made unit testing of the lookup(key) method
clumsy, because I have to clean it after each unit test. I refactored
it as:

class Lookup:
def __init__(self):
self.cache = {}

def lookup(key):
if key in self.cache:
return self.cache[key]

value = None

cmd = f"mycmd {key}"
proc = subprocess(cmd, capture_output=True, text=True, check=False)
if proc.returncode == 0:
value = proc.stdout.strip()
else:
logger.error("cmd returned error")

self.cache[key] = value
return value

Now it is easier to unit test, and the cache is not global. However, I
cannot instantiate Lookup inside the while- or for- loops in main(),
because the cache should be only one. I need to ensure there is only
one instance of Lookup - this is why I made it a global variable, so
that it is accessible to all functions in that script and the one that
actually needs it is 4 levels down in the call stack.

I have never done that in Python because I deliberately avoided such
complicated situations up to now. I know about the Singleton pattern,
but I have never implemented it in Python and I don't know if it is
Pythonish.

I am looking for the same behaviour as logging.getLogger(name).
logging.getLogger("myname") will always return the same object no
matter where it is called as long as the name argument is the same.

How would you advise me to implement that?

Regards
rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question about logging.config.dictConfig

[Ivan "Rambius" Ivanov]

On Tue, Feb 7, 2023 at 7:35 PM Peter J. Holzer  wrote:
>
> On 2023-02-07 17:58:26 -0500, Ivan "Rambius" Ivanov wrote:
> > I am trying to configure my loggers using dictConfig, but they do not
> > print anything. Here are more details.
> [...]
> > from myloggingconf import configure_logging
> >
> > logger = logging.getLogger(os.path.basename(__file__))
> [...]
> >
> > def main():
> > configure_logging()
> > dosmth()
> [...]
> > def configure_logging():
> > config = {
> > 'version': 1,
> > 'disable_existing_loggers': True,
>
>   'disable_existing_loggers': False,

Thank you! That helped a lot!

Regards
rambius

>
> I think I just found out why I've been cargo-culting this line since my
> early Django days ;-)-
>
> If you call getLogger very early (as you do), you don't want it disabled
> by a later call to dictConfig().
>
>
> > 'formatters': {
> > 'default': {
> > 'fmt': '%(asctime)s %(levelname)-7s %(name)s %(funcName)s 
> > %(message)s'
>   'format'
> > }
> > },
> [...]
> > }
> > logging.config.dictConfig(config)
> >
> > When I run uselog.py it prints nothing. I am wondering what is wrong
> > with the second configuration.
>
> See above.
>
> hp
>
> --
>_  | Peter J. Holzer| Story must make more sense than reality.
> |_|_) ||
> | |   | h...@hjp.at |-- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |   challenge!"
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Question about logging.config.dictConfig

[Ivan "Rambius" Ivanov]

Hello,

I am trying to configure my loggers using dictConfig, but they do not
print anything. Here are more details.

I have several python scripts that use a similar logging setup. I put
the common configuration in a separate module myloggingconf.py:

# myloggingconf.py
import logging

def configure_logging():
default = logging.Formatter('%(asctime)s %(levelname)-7s %(name)s
%(funcName)s %(message)s')

consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(default)

rootLogger = logging.getLogger()
rootLogger.addHandler(consoleHandler)
rootLogger.setLevel(logging.INFO)

Then I use it in the other scripts as:

# uselog.py
import logging
import os

from myloggingconf import configure_logging

logger = logging.getLogger(os.path.basename(__file__))

def dosmth():
logger.debug("debug")
logger.info("info")
logger.warning("warning")
logger.error("error")

def main():
configure_logging()
dosmth()

if __name__ == '__main__':
main()

This works correctly and the log messages about INFO print out. Now I
want to try dictConfig. I changed myloggingconf.configure_logging to:

# myloggingconf.py
import logging.config

def configure_logging():
config = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'default': {
'fmt': '%(asctime)s %(levelname)-7s %(name)s
%(funcName)s %(message)s'
}
},
'handlers': {
'stdout': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'default',
}
},
'loggers': {
'': {
'handlers': ['stdout'],
'level': 'DEBUG'
}
}
}
logging.config.dictConfig(config)

When I run uselog.py it prints nothing. I am wondering what is wrong
with the second configuration. I will appreciate any help.

Regards
rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Custom help format for a choice argparse argument

[Ivan "Rambius" Ivanov]

Hello Cameron,

On Fri, Jan 27, 2023 at 4:45 PM Cameron Simpson  wrote:
>
> On 27Jan2023 15:31, Ivan "Rambius" Ivanov  
> wrote:
> >I am developing a script that accepts a time zone as an option. The
> >time zone can be any from pytz.all_timezones. I have
> >
> >def main():
> >parser = argparse.ArgumentParser()
> >parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
> [...]
> >
> >It works, but when I run it with the -h option it dumps all entries in
> >pytz.all_timezones.
>
> What happens if you just presupply a `help=` parameter in
> `add_argument`?

I tried with def main():
parser = argparse.ArgumentParser()
parser.add_argument("-z", "--zone", choices=pytz.all_timezones,
help="a timezone from pytz.all_timezones")
args = parser.parse_args()
print(args)

-h still shows all the contents of pytz.all_timezones.

Regards
rambius

>
> Cheers,
> Cameron Simpson 
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Custom help format for a choice argparse argument

[Ivan "Rambius" Ivanov]

Hello,

On Fri, Jan 27, 2023 at 4:22 PM Weatherby,Gerard  wrote:
>
> Why not something like:
>
>
> parser.add_argument("-z", "--zone")
>
>args = parser.parse_args()
>if args.zone and args.zone not in ptyz.all_timezones:
>
> print(“Invalid timezone”,file=sys.stderr)
>
This is what I use now. I still wonder if I can mold HelpFormatter to
do what I want it to do.

>     …
>
>
>
>
> From: Python-list  on 
> behalf of Ivan "Rambius" Ivanov 
> Date: Friday, January 27, 2023 at 3:33 PM
> To: Python 
> Subject: Custom help format for a choice argparse argument
>
> *** Attention: This is an external email. Use caution responding, opening 
> attachments or clicking on links. ***
>
> Hello,
>
> I am developing a script that accepts a time zone as an option. The
> time zone can be any from pytz.all_timezones. I have
>
> def main():
> parser = argparse.ArgumentParser()
> parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
> args = parser.parse_args()
> print(args)
> print(f"Specified timezone: {args.zone}")
>
> It works, but when I run it with the -h option it dumps all entries in
> pytz.all_timezones. I would like to modify the help format for just
> -z|--zone option. I read the docs about HelpFormatter and argparse.py
> and I ended up with
>
> class CustomHelpFormatter(argparse.HelpFormatter):
> def _metavar_formatter(self, action, default_metavar):
> if action.dest == 'zone':
> result = 'zone from pytz.all_timezones'
> def format(tuple_size):
> if isinstance(result, tuple):
> return result
> else:
> return (result, ) * tuple_size
> return format
> else:
> return super(CustomHelpFormatter,
> self)._metavar_formatter(action, default_metavar)
>
>
> def main():
> parser = argparse.ArgumentParser(formatter_class=CustomHelpFormatter)
> parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
> args = parser.parse_args()
> print(args)
> print(f"Specified timezone: {args.zone}")
>
> This works, but is there a more elegant way to achieve it?
>
> Regards
> rambius
>
> --
> Tangra Mega Rock: 
> https://urldefense.com/v3/__http://www.radiotangra.com__;!!Cn_UX_p3!kiJusdm5pCptP3sOBX85KXqUJkqr2jSa4C_-WAqND7WkL-aw3BYbW50td_AcuzJ1XUPYIVO3JiLMc4gRWS885vTKFsFvaQ$
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kiJusdm5pCptP3sOBX85KXqUJkqr2jSa4C_-WAqND7WkL-aw3BYbW50td_AcuzJ1XUPYIVO3JiLMc4gRWS885vRXq-JKLg$



-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Custom help format for a choice argparse argument

[Ivan "Rambius" Ivanov]

Hello,

I am developing a script that accepts a time zone as an option. The
time zone can be any from pytz.all_timezones. I have

def main():
parser = argparse.ArgumentParser()
parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
args = parser.parse_args()
print(args)
print(f"Specified timezone: {args.zone}")

It works, but when I run it with the -h option it dumps all entries in
pytz.all_timezones. I would like to modify the help format for just
-z|--zone option. I read the docs about HelpFormatter and argparse.py
and I ended up with

class CustomHelpFormatter(argparse.HelpFormatter):
def _metavar_formatter(self, action, default_metavar):
if action.dest == 'zone':
result = 'zone from pytz.all_timezones'
def format(tuple_size):
if isinstance(result, tuple):
return result
else:
return (result, ) * tuple_size
return format
else:
return super(CustomHelpFormatter,
self)._metavar_formatter(action, default_metavar)


def main():
parser = argparse.ArgumentParser(formatter_class=CustomHelpFormatter)
parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
args = parser.parse_args()
print(args)
print(f"Specified timezone: {args.zone}")

This works, but is there a more elegant way to achieve it?

Regards
rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Resolving Weekday Schedules to Dates

[Ivan "Rambius" Ivanov]

Hello,

Do you know of a library that resolves schedules like every Wednesday
at 3:00pm to absolute time, that is return the datetime of the next
occurrence?

Regards
rambius

P.S.

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Concatenating a Hash to a String

[Ivan "Rambius" Ivanov]

Hello,

Thank you all for your help.

On Tue, Dec 1, 2020 at 1:38 PM MRAB  wrote:
> The bytes are all in the ASCII range, so you can convert it into a
> string using .decode('ascii').
I utilized encode and decode string methods to convert from bytes to strings

> And, of course, use parametrised queries, as ChrisA said.
I am using sqlite3 and it supports parameterized queries. Overall they
are better than constructing the sql statements manually.

Regards and thanks
rambius


-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Concatenating a Hash to a String

[Ivan "Rambius" Ivanov]

On Tue, Dec 1, 2020 at 12:39 AM Chris Angelico  wrote:
> Don't do this! DO NOT do this! Even if it might happen to work with a
> base 64 encoded value, this is a terrible terrible bug just waiting to
> happen. Instead, use *parameterized queries* and keep your SQL safe.

OK. What are parameterized queries? Can you give an example?

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Concatenating a Hash to a String

[Ivan "Rambius" Ivanov]

Hello,

I want to store the hashes of strings in a database and I have
problems generating the sql statements. I generate the hashes using
hashlib and then convert it to base64 and I put the base64
representation in the sql. Here is the code:

#!/usr/bin/env python3.8

import base64
import hashlib

def gen_sql(s):
  sha = hashlib.sha256()
  # need to encode s as binary, otherwise the next line throws
  # TypeError: Unicode-objects must be encoded before hashing
  sha.update(b"{s}")
  ehash = base64.b64encode(sha.digest())
  sql = "insert into HASHES value ('" + ehash + "')"

s = "test"
gen_sql(s)

This code throws

$ ./hashinstr.py
Traceback (most recent call last):
  File "./hashinstr.py", line 16, in 
gen_sql(s)
  File "./hashinstr.py", line 13, in gen_sql
sql = "insert into HASHES value ('" + ehash + "')"
TypeError: can only concatenate str (not "bytes") to str

Any help on how to concatenate ehash to the sql?

Regards
rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: EnvironmentError

[Ivan "Rambius" Ivanov]

Hello,

On Thu, Nov 19, 2020 at 1:56 PM Usman Musa  wrote:
>
> When I try to install a package or upgrade pip,  using pip install I got
> this error massage.
>  WARNING: Retrying (Retry(total=4, connect=None, read=None,
> redirect=None, status=None)) after connection broken by
> 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
> certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))':
> /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl
>   WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None,
> status=None)) after connection broken by
> 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
> certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))':
> /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl
>   WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None,
> status=None)) after connection broken by
> 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
> certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))':
> /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl
>   WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None,
> status=None)) after connection broken by
> 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
> certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))':
> /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl
>   WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None,
> status=None)) after connection broken by
> 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
> certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))':
> /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl
> ERROR: Could not install packages due to an EnvironmentError:
> HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries
> exceeded with url:
> /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl
> (Caused by SSLError(SSLCertVerificationError(1, '[SSL:
> CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate is not
> yet valid (_ssl.c:1122)')))
> Please help me out
> --
> https://mail.python.org/mailman/listinfo/python-list

The error says that the certificate of files.pythonhosted.org is not
valid, but this is definitely not the case. I think you sit behind
some ssl proxy and it behaves as a man-in-the-middle and replaces the
certificate with its own (I was experiencing that in a previous job).

You have several choices:

1) pass --trusted-host files.pythonhosted.org to disable the certificate check
2) import the ssl certificate of the ssl proxy (if you have such) into
python's certificate store
3) speak with the local system administrators about that. They may be of help.

Regards
rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: answer not correct

[Ivan "Rambius" Ivanov]

Hello,

First of all, remove the asterisks around the snippet, it makes it so
hard to copy and paste your code. My answer is inlined.

On Sun, Nov 8, 2020 at 2:28 PM Quentin Bock  wrote:
>
> *def add(numbers):*
> *   total = 1*
If this is your sum, you need to initialize it to zero:

total = 0

> *   for x in numbers:*
> *  total += 1*
Do you mean total += x?

Regards
rambius

> *   return total*
> *print(add[1999, -672, 64]))*
>
> *the answer I get is 4 but it should be 1,411*
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: System Beep?

[Ivan "Rambius" Ivanov]

Hello,

On Fri, Mar 8, 2019, 3:19 PM Steve  wrote:

> = RESTART: C:\Gork\Med Insulin codes\MedReminder 127.py
> =
> Traceback (most recent call last):
>   File "C:\Gork\Med Insulin codes\MedReminder 127.py", line 13, in
> 
> windsound.Beep  #(frequency, duration)
> NameError: name 'windsound' is not defined
>
> Something missing in my code?
>

You have a typo. It is winsound, not winDsound.

Regards
rambius


> 
> 98% of lawyers give the other 2% a bad name.
>
>
> -Original Message-
> From: Python-list  On
> Behalf Of David Raymond
> Sent: Friday, March 8, 2019 3:02 PM
> To: python-list@python.org
> Subject: RE: System Beep?
>
> Windows has the built in winsound module that lets you set frequency and
> duration, not sure about other OS's.
>
> https://docs.python.org/3.7/library/winsound.html
>
>
> -Original Message-
> From: Python-list
> [mailto:python-list-bounces+david.raymond=tomtom@python.org] On Behalf
> Of Steve
> Sent: Friday, March 08, 2019 1:14 PM
> To: python-list@python.org
> Subject: System Beep?
>
> How can I cause a system beep using code?
> This is using the computer's internal speaker, not an audio external
> speaker.
> Do I have control of duration or volume?
>
> Steve
>
>
>
>
>
> Footnote:
> Every minute, every single day, the equivalent of a truckload of plastic
> enters our oceans.
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
> --
> https://mail.python.org/mailman/listinfo/python-list
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: what type of application implemented with python?

[Ivan "Rambius" Ivanov]

Hello,

Python's own website contains a non-exhaustive list of applications
implemeted in Python: https://www.python.org/about/apps/

Regards
Rambius

On Sat, Apr 15, 2017 at 4:13 PM, kondaiah sinha  wrote:
> what type of application implemented with python?...like by using java we can 
> implement web,Standard alone app's,mobile based.
> Can anyone tell me is it possible to implement Standard alone applications 
> with, if possible give any example
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Getting stdout and stderr from subprocess in correct order

[Ivan "Rambius" Ivanov]

Hello,

Thank you all for your suggestions. I will see what will apply to my use case.

Regards
Rambius

On Sat, Mar 4, 2017 at 5:37 PM, Piet van Oostrum
<pie...@pietvanoostrum.com> wrote:
> "Ivan \"Rambius\" Ivanov" <rambiusparkisan...@gmail.com> writes:
>
>> Dear colleagues,
>>
>> I using subprocess module and I am wondering how I can get the output
>> of the spawned process's stdout and stderr in the right order. Here
>> are my sample programs:
>>
>> $ cat subprc.py
>> import subprocess
>> import sys
>>
>> f = 'hw.py'
>> p = subprocess.run([sys.executable, f], stdout=subprocess.PIPE,
>> stderr=subprocess.PIPE)
>> print(p.stdout)
>> print(p.stderr)
>>
>> $ cat hw.py
>> import sys
>>
>> print("a", file=sys.stdout)
>> print("b", file=sys.stderr)
>> print("c", file=sys.stdout)
>> print("d", file=sys.stderr)
>>
>> When i run hw.py I get
>>
>> $ ./hw.py
>> a
>> b
>> c
>> d
>>
>> When I run it through subprc.py, I do get standard out and standard
>> errors streams, but they are separated and not in the order above:
>>
>> $ ./subprc.py
>> b'a\nc\n'
>> b'b\nd\n'
>>
>> How should I use subprocess in order to get the outputs in the correct
>> order? Thank you for your help in advance.
>>
> If you want them in the order they are produced then you have to put
> them through a single channel with stderr=subprocess.STDOUT.
>
> p = subprocess.run([sys.executable, f], stdout=subprocess.PIPE,
> stderr=subprocess.STDOUT, encoding='ascii')
>
> I added an encoding so that it will be read as a text file rather than
> binary. And of course you should read only stdout (stderr will give
> None)
>
> Moreover you must make sure that each write is flushed. This can be done
> in three ways:
>
> 1. Add the flush=True argument to the print statements.
> print("a", file=sys.stdout, flush=True)
> print("b", file=sys.stderr, flush=True)
> print("c", file=sys.stdout, flush=True)
> print("d", file=sys.stderr, flush=True)
>
> 2. Change the files to a line buffered version.
> import os
> sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 1)
> sys.stderr = os.fdopen(sys.stderr.fileno(), 'w', 1)
>
> 3. Add a flush() call after each print statement, like
> sys.stdout.flush()
>
> Then you get them in order.
> $ python3 subprc.py
> a
> b
> c
> d
>
> Of course you won't be able to tell from which stream each line comes.
> The streams are byte streams, not message streams. You would have to put
> extra information, e.g. a prefix, in your print statements.
> --
> Piet van Oostrum <pie...@pietvanoostrum.com>
> WWW: http://pietvanoostrum.com/
> PGP key: [8DAE142BE17999C4]
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Getting stdout and stderr from subprocess in correct order

[Ivan "Rambius" Ivanov]

Dear colleagues,

I using subprocess module and I am wondering how I can get the output
of the spawned process's stdout and stderr in the right order. Here
are my sample programs:

$ cat subprc.py
import subprocess
import sys

f = 'hw.py'
p = subprocess.run([sys.executable, f], stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
print(p.stdout)
print(p.stderr)

$ cat hw.py
import sys

print("a", file=sys.stdout)
print("b", file=sys.stderr)
print("c", file=sys.stdout)
print("d", file=sys.stderr)

When i run hw.py I get

$ ./hw.py
a
b
c
d

When I run it through subprc.py, I do get standard out and standard
errors streams, but they are separated and not in the order above:

$ ./subprc.py
b'a\nc\n'
b'b\nd\n'

How should I use subprocess in order to get the outputs in the correct
order? Thank you for your help in advance.

Regards
Rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com
-- 
https://mail.python.org/mailman/listinfo/python-list