Re: In code, list.clear doesn't throw error - it's just ignored

[Dan Stromberg]

On Sun, Nov 13, 2022 at 4:45 PM DFS  wrote:

> In code, list.clear is just ignored.
> At the terminal, list.clear shows
> 
>
>
> in code:
> x = [1,2,3]
> x.clear
> print(len(x))
> 3
>
> at terminal:
> x = [1,2,3]
> x.clear
> 
> print(len(x))
> 3
>
>
> Caused me an hour of frustration before I noticed list.clear() was what
> I needed.
>
> x = [1,2,3]
> x.clear()
> print(len(x))
> 0
>
> --
> https://mail.python.org/mailman/listinfo/python-list



I'm not 100% sanguine about properties, but the fact is they are part of
the language:

$ cat p
below cmd output started 2022 Fri Nov 25 07:54:42 AM PST
#!/usr/bin/env python3

class P:
def __init__(self):
self.count = 0

@property
def increment(self):
self.count += 1

def result(self):
return self.count


p = P()
p.increment
p.increment
print(p.result())
above cmd output done2022 Fri Nov 25 07:54:42 AM PST
dstromberg@tp-mini-c:~/src/experiments/property x86_64-pc-linux-gnu 2670

$ ./p
below cmd output started 2022 Fri Nov 25 07:54:44 AM PST
2

As you can see, if the interpreter refused to do something with p.increment
because it has no parens, the meaning of this code would change
significantly.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are these good ideas?

[Dan Stromberg]

On Mon, Nov 14, 2022 at 11:33 AM Axy via Python-list 
wrote:

> On 14/11/2022 17:14, Stephen Tucker wrote:
> > Hi,
> >
> > I have two related issues I'd like comments on.
> >
> > Issue 1 - Global Values
>
> Your "global variables" module acts exactly as a singleton class.
>

Which is apparently a design pattern that some now believe is regrettable.

It's something I've done before too, but it's pretty much shared, mutable
state, which isn't great for functional programming or for parallelism.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: str.replace() when str contains \

[Dan Stromberg]

I believe you would do well to print a, before trying to transform it into
something else.

'\2' is chr(2).
'\a' is the bell character, and is unprintable.
'\_' is two characters though.

On Sat, Oct 29, 2022 at 2:12 PM Bernard LEDRU 
wrote:

> Hello,
>
> https://docs.python.org/3/library/stdtypes.html#string-methods
>
> str.replace(old, new[, count])¶
> Return a copy of the string with all occurrences of substring old
> replaced by new. If the optional argument count is given, only the first
> count occurrences are replaced.
>
> Attention when the string contains the escape character.
>
> Consider :
>
> >>> a="H:\2023"; print(a.replace("\\","/"))
> H:3
> >>> a="H:\a2023"; print(a.replace("\\","/"))
> H:2023
> >>> a="H:\_2023"; print(a.replace("\\","/"))
> H:/_2023
>
> Best regards,
> Bernard LEDRU
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Typing: Is there a "cast operator"?

[Dan Stromberg]

On Sun, Oct 23, 2022 at 2:11 PM Paulo da Silva <
p_d_a_s_i_l_v_a...@nonetnoaddress.pt> wrote:

> Hello!
>
> I am in the process of "typing" of some of my scripts.
> Using it should help a lot to avoid some errors.
> But this is new for me and I'm facing some problems.
>
> Let's I have the following code (please don't look at the program content):
>
> f=None  # mypy naturally assumes Optional(int) because later, at open,
> it is assigned an int.
> ..
> if f is None:
> f=os.open(...
> ..
> if f is not None:
> os.write(f, ...)
> ..
> if f is not None:
> os.close(f)
>
> When I use mypy, it claims
> Argument 1 to "write" has incompatible type "Optional[int]"; expected "int"
> Argument 1 to "close" has incompatible type "Optional[int]"; expected "int"
>
> How to solve this?
> Is there a way to specify that when calling os.open f is an int only?
>
> I use None a lot for specify uninitialized vars.
>

I've found that mypy understands simple assert statements.

So if you:
if f is not None:
assert f is not None
os.write(f, ...)

You might be in good shape.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A trivial question that I don't know - document a function/method

[Dan Stromberg]

I don't think there is a "correct" way.  It depends somewhat on what tools
you're using.  I like pydocstyle, which I have hung off of vim with
syntastic.  pydocstyle checks for https://peps.python.org/pep-0257/
conformance.

Also, rather than describe the types of formal parameters to functions in a
docstring, I like to use mypy for https://peps.python.org/pep-0484/ with
its --disallow-untyped-calls and --ignore-missing-imports options, which I
hang off of a Makefile, called by a vim macro.

On Sat, Oct 22, 2022 at 3:39 PM Paulo da Silva <
p_d_a_s_i_l_v_a...@nonetnoaddress.pt> wrote:

> Hi all!
>
> What is the correct way, if any, of documenting a function/method?
>
> 1.
> def foo(a,b):
> """ A description.
> a: Whatever 1
> b: Whatever 2
> """
> ...
>
> 2.
> def foo(a,b):
> """ A description.
> a -- Whatever 1
> b -- Whatever 2
> """
> ...
>
> 3.
> def foo(a,b):
> """ A description.
> @param a: Whatever 1
> @param b: Whatever 2
> """
> ...
>
> 4.
> def foo(a,b):
> """ A description.
> :param a: Whatever 1
> :param b: Whatever 2
> """
> ...
>
> 5.
> Any other ...
>
> Any comments/suggestions are welcome.
> Thanks.
> Paulo
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command [POSTPONED]

[Dan Stromberg]

On Wed, Oct 12, 2022 at 9:57 PM Cameron Simpson  wrote:

> On 13Oct2022 03:25, Paulo da Silva 
> wrote:
> >There is another problem involved. The script, works fine except when
> >launched by cron! Why?
>
> Record the script output:
>
>  # record all output
>  exec >/tmp/script.$$.out 2>&1
>  # dump the envionment
>  env | sort
>  # turn on execution tracing
>  set -x
>  ... rest of the script
>
> and have a look afterwards. Cron's environment is very minimal. This
> will show you what's in it.
>

Careful.  On some systems if someone restarts the cron daemon, it could
pick up a larger environment than after being started on boot.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Find the path of a shell command

[Dan Stromberg]

On Wed, Oct 12, 2022 at 11:13 AM Paulo da Silva <
p_d_a_s_i_l_v_a...@nonetnoaddress.pt> wrote:

> Hi!
>
> The simple question: How do I find the full path of a shell command
> (linux), i.e. how do I obtain the corresponding of, for example,
> "type rm" in command line?
>
> The reason:
> I have python program that launches a detached rm. It works pretty well
> until it is invoked by cron! I suspect that for cron we need to specify
> the full path.
> Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin?
> What about other commands?
>
I like to test my cronjobs with something like:

env - /usr/local/bin/my-cronjob

This will empty out the environment, and force you to set $PATH yourself.

Alternatively, you can "ps axfwwe" (on Linux) to see environment variables,
and check what the environment of cron (or similar) is.  It is this
environment (mostly) that cronjobs will inherit.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: flattening lists

[Dan Stromberg]

On Tue, Oct 11, 2022 at 12:48 PM SquidBits _  wrote:

> Does anyone else think there should be a flatten () function, which just
> turns a multi-dimensional list into a one-dimensional list in the order
> it's in. e.g.
>
> [[1,2,3],[4,5,6,7],[8,9]] becomes [1,2,3,4,5,6,7,8,9].
>
> I have had to flatten lists quite a few times and it's quite tedious to
> type out. It feels like this should be something built in to python, anyone
> else think this way?
>

I think the usual argument against putting something like this in the
standard library (I hope it won't be built in), is that there are many ways
to define "flatten".  That is, should it only go one level deep?   All the
way to the bottom?  n levels deep?  Should it do something special with
lists, dicts, tuples, sets?

This looks like a nice URL on the topic:
https://www.pythonpool.com/flatten-list-python/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: for -- else: what was the motivation?

[Dan Stromberg]

The else is executed if you don't "break" out of the loop early.

It cuts down on boolean flags.

On Fri, Oct 7, 2022 at 8:40 PM Axy via Python-list 
wrote:

> Hi there,
>
> this is rather a philosophical question, but I assume I miss something.
> I don't remember I ever used else clause for years I was with python and
> my expectation was it executed only if the the main body was never run.
> Ha-ha! I was caught by this mental trap.
>
> So, seriously, why they needed else if the following pieces produce same
> result? Does anyone know or remember their motivation?
>
> Just curious.
>
> Axy.
>
> print('--- with else')
>
>
> for i in [1,2,3]:
>  print(i)
> else:
>  print(4)
>
> for i in []:
>  print(i)
> else:
>  print(5)
>
> print('--- without else')
>
> for i in [1,2,3]:
>  print(i)
> print(4)
>
> for i in []:
>  print(i)
> print(5)
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.9.14

[Dan Stromberg]

‪On Wed, Sep 14, 2022 at 6:05 AM ‫אורי‬‎  wrote:‬

> Hi,
>
> Python 3.9.14 has been released on Sept. 6, 2022. As I can see written on
> https://www.python.org/downloads/release/python-3914/:
>
> According to the release calendar specified in PEP 596, Python 3.9 is now
> in the "security fixes only" stage of its life cycle: the 3.9 branch only
> accepts security fixes and releases of those are made irregularly in
> source-only form until October 2025. Python 3.9 isn't receiving regular bug
> fixes anymore, and binary installers are no longer provided for it. Python
> 3.9.13 was the last full bugfix release of Python 3.9 with binary
> installers.
>
>
> Is there a safe way to install a 64-bit version of Python 3.9.14 on
> Windows?
>

I use Windows as little as I can, but you could check to see if Conda or
the Microsoft Store will give you CPython 3.9 binaries.

If that isn't happening, you could try compiling your own from the
python.org sources.  On Linux, that's pretty simple: extract the archive,
./configure && make && sudo make install
I don't know if it'll be easier or harder on WIndows.  I want to say you
could get MSYS2 as a development environment, but ISTR hearing that Python
wants to be compiled with Microsoft's compiler on Windows for ABI
compatibility.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace an instance method?

[Dan Stromberg]

On Fri, Sep 16, 2022 at 2:06 PM Ralf M.  wrote:

> I would like to replace a method of an instance, but don't know how to
> do it properly.
>

You appear to have a good answer, but...  are you sure this is a good idea?

It'll probably be confusing to future maintainers of this code, and I doubt
static analyzers will like it either.

I'm not the biggest fan of inheritance you'll ever meet, but maybe this is
a good place for it?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Local variable definition in Python list comprehension

[Dan Stromberg]

On Thu, Sep 1, 2022 at 9:16 AM Chris Angelico  wrote:

> On Fri, 2 Sept 2022 at 02:10, James Tsai  wrote:
> >
> > Hello,
> >
> > I find it very useful if I am allowed to define new local variables in a
> list comprehension. For example, I wish to have something like
> > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or
> > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80].
> >
> > For now this functionality can be achieved by writing
> > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80].
> >
> > Is it worthwhile to add a new feature like this in Python? If so, how
> can I propose this to PEP?
>
> Not everything has to be a one-liner.
>
So true!

I like list comprehensions and generator expressions, but sometimes I end
up regretting their use when there's a bug, and I have to convert one to a
for loop + list.append in order to debug.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Coffee

[Dan Stromberg]

On Mon, Aug 29, 2022 at 1:10 PM Meredith Montgomery 
wrote:

> r...@zedat.fu-berlin.de (Stefan Ram) writes:
>
> > |Python's obviously a great tool for all kinds of programming things,
> > |and I would say if you're only gonna use one programming
> > |language in your live, Python will probably the right one.
> > Brian Kernighan
> >
> >   I transcribed this from the recent video
> >   "Coffee with Brian Kernighan".
>
> Sounds reasonable.  I have been learning Python bit by bit simply
> because there seems to be no other way to talk to university people.
> But somehow I am so in love with Lisp that it makes me sort of blind
> because sometimes I feel more productive in Python simply because I'm
> always using it.  When I can write Lisp, I do it, but often I feel like
> I'm a role-playing TCP Slow Start or something like that.
>

ISTR hearing that Python and Lisp are pretty similar semantically - not
because Python copied it, but because similar thinking went into the design
of each.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: subprocess.popen how wait complete open process

[Dan Stromberg]

On Sun, Aug 21, 2022 at 2:05 PM Chris Angelico  wrote:

> On Mon, 22 Aug 2022 at 05:39, simone zambonardi
>  wrote:
> >
> > Hi, I am running a program with the punishment subrocess.Popen(...) what
> I should do is to stop the script until the launched program is fully open.
> How can I do this? I used a time.sleep() function but I think there are
> other ways. Thanks
> >
>
> First you have to define "fully open". How would you know?
>

If you're on X11, you could conceivably use:
 xwininfo -tree -root
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: setup.py + cython == chicken and the egg problem

[Dan Stromberg]

On Tue, Aug 16, 2022 at 2:03 PM Dan Stromberg  wrote:

> Hi folks.
>
> I'm attempting to package up a python package that uses Cython.
>
> Rather than build binaries for everything under the sun, I've been
> focusing on including the .pyx file and running cython on it at install
> time.  This requires a C compiler, but I'm OK with that.
>
> BTW, the pure python version works fine, and the cython version works too
> as long as you preinstall cython - but I don't want users to have to know
> that :)
>

For the actual chicken-and-egg problem, I'd needed to include my
pyproject.toml in my MANIFEST.in, like:
include pyx_treap.pyx pyx_treap.c pyproject.toml
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: setup.py + cython == chicken and the egg problem

[Dan Stromberg]

On Wed, Aug 17, 2022 at 3:05 PM Dan Stromberg  wrote:

> I commented out those too lines, but I'm still getting errors.  They seem
>> to stem from:
>> $ "/home/dstromberg/venv/pyx-treap-testing/bin/python3",
>> ["/home/dstromberg/venv/pyx-treap-testing/bin/python3",
>> "/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py",
>> "install", "--ignore-installed", "--no-user", "--prefix",
>> "/tmp/pip-build-env-9_ivrsb6/overlay", "--no-warn-script-location",
>> "--no-binary", ":none:", "--only-binary", ":none:", "-i", "
>> https://test.pypi.org/simple/;, "--", "setuptools >= 44.1.1", "wheel",
>> "Cython"]
>> "/home/dstromberg/venv/pyx-treap-testing/bin/python3"
>> "/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py"
>> "install" "--ignore-installed" "--no-user" "--prefix"
>> "/tmp/pip-build-env-9_ivrsb6/overlay" "--no-warn-script-location"
>> "--no-binary" ":none:" "--only-binary" ":none:" "-i" "
>> https://test.pypi.org/simple/; "--" "setuptools >= 44.1.1" "wheel"
>> "Cython"
>> Looking in indexes: https://test.pypi.org/simple/
>> ERROR: Could not find a version that satisfies the requirement
>> setuptools>=44.1.1 (from versions: none)
>> ERROR: No matching distribution found for setuptools>=44.1.1
>>
>> I copied that out of an strace.
>>
>> That's likely related to my pyproject.toml:
>> $ cat pyproject.toml
>> below cmd output started 2022 Wed Aug 17 01:57:09 PM PDT
>> [build-system]
>> requires = ["setuptools >= 44.1.1", "wheel", "Cython"]
>> build-backend = "setuptools.build_meta"
>>
>> Any other suggestions folks?
>>
>
> I don't know why, but if I delete the --ignore-installed option, I don't
> get the error:
> $ "/home/dstromberg/venv/pyx-treap-testing/bin/python3"
> "/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py"
> "install" "--ignore-installed" "--prefix"
> "/tmp/pip-build-env-9_ivrsb6/overlay" "-i" "https://test.pypi.org/simple/;
> "--" "setuptools>=44.1.1"
> below cmd output started 2022 Wed Aug 17 02:56:24 PM PDT
> Looking in indexes: https://test.pypi.org/simple/
> ERROR: Could not find a version that satisfies the requirement
> setuptools>=44.1.1 (from versions: none)
> ERROR: No matching distribution found for setuptools>=44.1.1
> (setuptools-investigation) above cmd output done2022 Wed Aug 17
> 02:56:24 PM PDT
> dstromberg@tp-mini-c:~ x86_64-pc-linux-gnu 2995
>
> $ "/home/dstromberg/venv/pyx-treap-testing/bin/python3"
> "/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py"
> "install" "--prefix" "/tmp/pip-build-env-9_ivrsb6/overlay" "-i" "
> https://test.pypi.org/simple/; "--" "setuptools>=44.1.1"
> below cmd output started 2022 Wed Aug 17 02:56:35 PM PDT
> Looking in indexes: https://test.pypi.org/simple/
> Requirement already satisfied: setuptools>=44.1.1 in
> ./venv/pyx-treap-testing/lib/python3.9/site-packages (63.4.1)
>
> If I search for foo on pypi and testpypi, shouldn't I get foo before
> foo-bar and bar-foo?
>
> Because if I search for setuptools on pypi, I get setuptools as my first
> hit, but on testpypi, I don't see setuptools anywhere in the first page.
> That's perhaps significant if the search ordering is working as described
> above.
>
> ?
>

IOW, maybe testpypi doesn't have setuptools.  Indeed, it doesn't appear
to.  Publishing and installing from the real pypi seems to be working.

Thanks folks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: setup.py + cython == chicken and the egg problem

[Dan Stromberg]

On Wed, Aug 17, 2022 at 1:58 PM Dan Stromberg  wrote:

> On Wed, Aug 17, 2022 at 10:20 AM Christian Gollwitzer 
> wrote:
>
>> Am 16.08.22 um 23:03 schrieb Dan Stromberg:
>> > I'm attempting to package up a python package that uses Cython.
>> >
>> > Rather than build binaries for everything under the sun, I've been
>> focusing
>> > on including the .pyx file and running cython on it at install time.
>> This
>> > requires a C compiler, but I'm OK with that.
>> >
>> > However, when I try to install the package from test.pypi.org, I get:
>> > $ python3 -m pip install -i https://test.pypi.org/simple/ pyx-treap
>> > below cmd output started 2022 Tue Aug 16 01:55:16 PM PDT
>> > Looking in indexes: https://test.pypi.org/simple/
>> > Collecting pyx-treap
>> >Downloading
>> >
>> https://test-files.pythonhosted.org/packages/3a/41/af5360934adccfc086a39e1f720323895144b53454ff6dacc0f06267db55/pyx_treap-2.0.15.tar.gz
>> > (125 kB)
>> >
>> >
>>  
>> 
>> > 125.9/125.9 kB 1.9 MB/s eta 0:00:00
>> >Installing build dependencies ... error
>> >error: subprocess-exited-with-error
>> >
>> >×? pip subprocess to install build dependencies did not run
>> successfully.
>> >?? exit code: 1
>> >> [3 lines of output]
>> >Looking in indexes: https://test.pypi.org/simple/
>> >ERROR: Could not find a version that satisfies the requirement
>> > setuptools (from versions: none)
>> >ERROR: No matching distribution found for setuptools
>> >[end of output]
>> >
>> >note: This error originates from a subprocess, and is likely not a
>> > problem with pip.
>> > error: subprocess-exited-with-error
>>
>>
>> I looked at your code and I think you are trying too hard. As far as I
>> understand, you need Cython to be installed before the build process
>> begins. Your entry in pyproject.toml should take care of that.
>> But you also have these lines in your setup.py
>>
>> subprocess.check_call('%s -m pip install cython' % (sys.executable, ),
>> shell=True)
>> subprocess.check_call('%s -m cython pyx_treap.pyx' % (sys.executable, ),
>> shell=True)
>>
>> The first one calls out to pip while pip is already running, I'm not
>> sure that this will work, but judging from the error message it is
>> looking for the requirements also from test.pypi. Maybe this is the
>> reason that it fails (the error message says that it can't find
>> setuptools). So jut delete this line and it might already work
>>
>> The second line, which compiles the Cython code, also runs *at every
>> invocation of setup.py*, even if you'd do just
>>
>> python3 setup.py --help
>>
>> It may still work, but the correct way to do it is to create a build
>> extension for setuptools. In my project you can see this here:
>>
>> https://github.com/j-from-b/CDEF/blob/main/setup.py#L88
>>
>> OTOH, I would be surprised if Cython did not have this already, indeed
>> you imported cythonize from Cython.Build. So maybe just deleting these
>> two lines and it might work?
>>
>
> I commented out those too lines, but I'm still getting errors.  They seem
> to stem from:
> $ "/home/dstromberg/venv/pyx-treap-testing/bin/python3",
> ["/home/dstromberg/venv/pyx-treap-testing/bin/python3",
> "/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py",
> "install", "--ignore-installed", "--no-user", "--prefix",
> "/tmp/pip-build-env-9_ivrsb6/overlay", "--no-warn-script-location",
> "--no-binary", ":none:", "--only-binary", ":none:", "-i", "
> https://test.pypi.org/simple/;, "--", "setuptools >= 44.1.1", "wheel",
> "Cython"]
> "/home/dstromberg/venv/pyx-treap-testing/bin/python3"
> "/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py"
> "install" "--ignore-installed" "--no-user" "--prefix"
> "/tmp/pip-build-env-9_ivrsb6/overlay" "--no-warn-script-location"
> "--no-binary" ":none:" "--only-binary" ":none:" "-i" "
> https://test.pypi.org/simple/; "--" "setuptools >= 44.1.1" "

Re: setup.py + cython == chicken and the egg problem

[Dan Stromberg]

On Wed, Aug 17, 2022 at 10:20 AM Christian Gollwitzer 
wrote:

> Am 16.08.22 um 23:03 schrieb Dan Stromberg:
> > I'm attempting to package up a python package that uses Cython.
> >
> > Rather than build binaries for everything under the sun, I've been
> focusing
> > on including the .pyx file and running cython on it at install time.
> This
> > requires a C compiler, but I'm OK with that.
> >
> > However, when I try to install the package from test.pypi.org, I get:
> > $ python3 -m pip install -i https://test.pypi.org/simple/ pyx-treap
> > below cmd output started 2022 Tue Aug 16 01:55:16 PM PDT
> > Looking in indexes: https://test.pypi.org/simple/
> > Collecting pyx-treap
> >Downloading
> >
> https://test-files.pythonhosted.org/packages/3a/41/af5360934adccfc086a39e1f720323895144b53454ff6dacc0f06267db55/pyx_treap-2.0.15.tar.gz
> > (125 kB)
> >
> >
>  
> 
> > 125.9/125.9 kB 1.9 MB/s eta 0:00:00
> >Installing build dependencies ... error
> >error: subprocess-exited-with-error
> >
> >×? pip subprocess to install build dependencies did not run
> successfully.
> >?? exit code: 1
> >> [3 lines of output]
> >Looking in indexes: https://test.pypi.org/simple/
> >ERROR: Could not find a version that satisfies the requirement
> > setuptools (from versions: none)
> >ERROR: No matching distribution found for setuptools
> >[end of output]
> >
> >note: This error originates from a subprocess, and is likely not a
> > problem with pip.
> > error: subprocess-exited-with-error
>
>
> I looked at your code and I think you are trying too hard. As far as I
> understand, you need Cython to be installed before the build process
> begins. Your entry in pyproject.toml should take care of that.
> But you also have these lines in your setup.py
>
> subprocess.check_call('%s -m pip install cython' % (sys.executable, ),
> shell=True)
> subprocess.check_call('%s -m cython pyx_treap.pyx' % (sys.executable, ),
> shell=True)
>
> The first one calls out to pip while pip is already running, I'm not
> sure that this will work, but judging from the error message it is
> looking for the requirements also from test.pypi. Maybe this is the
> reason that it fails (the error message says that it can't find
> setuptools). So jut delete this line and it might already work
>
> The second line, which compiles the Cython code, also runs *at every
> invocation of setup.py*, even if you'd do just
>
> python3 setup.py --help
>
> It may still work, but the correct way to do it is to create a build
> extension for setuptools. In my project you can see this here:
>
> https://github.com/j-from-b/CDEF/blob/main/setup.py#L88
>
> OTOH, I would be surprised if Cython did not have this already, indeed
> you imported cythonize from Cython.Build. So maybe just deleting these
> two lines and it might work?
>

I commented out those too lines, but I'm still getting errors.  They seem
to stem from:
$ "/home/dstromberg/venv/pyx-treap-testing/bin/python3",
["/home/dstromberg/venv/pyx-treap-testing/bin/python3",
"/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py",
"install", "--ignore-installed", "--no-user", "--prefix",
"/tmp/pip-build-env-9_ivrsb6/overlay", "--no-warn-script-location",
"--no-binary", ":none:", "--only-binary", ":none:", "-i", "
https://test.pypi.org/simple/;, "--", "setuptools >= 44.1.1", "wheel",
"Cython"]
"/home/dstromberg/venv/pyx-treap-testing/bin/python3"
"/home/dstromberg/venv/pyx-treap-testing/lib/python3.9/site-packages/pip/__pip-runner__.py"
"install" "--ignore-installed" "--no-user" "--prefix"
"/tmp/pip-build-env-9_ivrsb6/overlay" "--no-warn-script-location"
"--no-binary" ":none:" "--only-binary" ":none:" "-i" "
https://test.pypi.org/simple/; "--" "setuptools >= 44.1.1" "wheel" "Cython"
Looking in indexes: https://test.pypi.org/simple/
ERROR: Could not find a version that satisfies the requirement
setuptools>=44.1.1 (from versions: none)
ERROR: No matching distribution found for setuptools>=44.1.1

I copied that out of an strace.

That's likely related to my pyproject.toml:
$ cat pyproject.toml
below cmd output started 2022 Wed Aug 17 01:57:09 PM PDT
[build-system]
requires = ["setuptools >= 44.1.1", "wheel", "Cython"]
build-backend = "setuptools.build_meta"

Any other suggestions folks?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: setup.py + cython == chicken and the egg problem

[Dan Stromberg]

On Tue, Aug 16, 2022 at 2:08 PM Chris Angelico  wrote:

> On Wed, 17 Aug 2022 at 07:05, Dan Stromberg  wrote:
> >
> > Hi folks.
> >
> > I'm attempting to package up a python package that uses Cython.
> >
> > Rather than build binaries for everything under the sun, I've been
> focusing
> > on including the .pyx file and running cython on it at install time.
> This
> > requires a C compiler, but I'm OK with that.
> >
>
> Is keeping the cythonized file an option? That would still require a C
> compiler, but wouldn't require Cython.
>
> ChrisA
>

That comes under "one of the many things I tried"; it seemed like some of
the code in the .c was tailored to the specific version of python it was
being installed on or something.

But if someone has seen this work well, I'm more than open to trying it
again.
-- 
https://mail.python.org/mailman/listinfo/python-list

setup.py + cython == chicken and the egg problem

[Dan Stromberg]

Hi folks.

I'm attempting to package up a python package that uses Cython.

Rather than build binaries for everything under the sun, I've been focusing
on including the .pyx file and running cython on it at install time.  This
requires a C compiler, but I'm OK with that.

However, when I try to install the package from test.pypi.org, I get:
$ python3 -m pip install -i https://test.pypi.org/simple/ pyx-treap
below cmd output started 2022 Tue Aug 16 01:55:16 PM PDT
Looking in indexes: https://test.pypi.org/simple/
Collecting pyx-treap
  Downloading
https://test-files.pythonhosted.org/packages/3a/41/af5360934adccfc086a39e1f720323895144b53454ff6dacc0f06267db55/pyx_treap-2.0.15.tar.gz
(125 kB)

 

125.9/125.9 kB 1.9 MB/s eta 0:00:00
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  ×? pip subprocess to install build dependencies did not run successfully.
  ?? exit code: 1
  > [3 lines of output]
  Looking in indexes: https://test.pypi.org/simple/
  ERROR: Could not find a version that satisfies the requirement
setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  [end of output]

  note: This error originates from a subprocess, and is likely not a
problem with pip.
error: subprocess-exited-with-error

×? pip subprocess to install build dependencies did not run successfully.
?? exit code: 1
> See above for output.

note: This error originates from a subprocess, and is likely not a problem
with pip.

But I analyzed the pip install with strace, and found no interesting
exec's, and no interesting "= E" patterns.

I've tried quite an assortment of things to get past this, including most
of those at:
https://stackoverflow.com/questions/4996589/in-setup-py-or-pip-requirements-file-how-to-control-order-of-installing-package
Except the one at:
https://stackoverflow.com/a/54269307/1084684
...because I was having a "there must be a better way" reaction to it.

/Is/ there a better way?

There're all these newer ways of doing packaging - surely one of them
addresses this problem?

The code I'm trying to package is at:
https://stromberg.dnsalias.org/svn/treap/trunk/cython

BTW, the pure python version works fine, and the cython version works too
as long as you preinstall cython - but I don't want users to have to know
that :)

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Parallel(?) programming with python

[Dan Stromberg]

Queues are better than lists for concurrency.  If you get the right kind,
they have implicit locking, making your code simpler and more robust at the
same time.

CPython threading is mediocre for software systems that have one or more
CPU-bound threads, and your FFT might be CPU-bound.

Rather than using threading directly, you probably should use
https://docs.python.org/3/library/concurrent.futures.html , which gives you
easy switching between threads and processes.

Or if you, like me, get inordinately joyous over programs that run on more
than one kind of Python, you could give up concurrent.futures and use
_thread.  Sadly, that gives up easy flipping between threads and processes,
but gives you easy flipping between CPython and micropython.  Better still,
micropython appears to have more scalable threading than CPython, so if you
decide you need 20 CPU-hungry threads someday, you are less likely to be in
a bind.

For reading from a socket, if you're not going the REST route, may I
suggest https://stromberg.dnsalias.org/~strombrg/bufsock.html ?  It deals
with framing and lengths relatively smoothly.  Otherwise, robust socket
code tends to need while loops and tedious arithmetic.

HTH

On Mon, Aug 8, 2022 at 10:59 AM Andreas Croci  wrote:

> I would like to write a program, that reads from the network a fixed
> amount of bytes and appends them to a list. This should happen once a
> second.
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Trying to understand nested loops

[Dan Stromberg]

On Fri, Aug 5, 2022 at 12:54 PM Grant Edwards 
wrote:

> In C, this doesn't do what it looks like it's supposed to do.
>
>if (foo)
>  do_this();
>  and_this();
>then_do_this();
>
It's been quite a while since I used C, but with the right compiler
flag(s), I think this may be a thing of the past when compiling with gcc:
https://developers.redhat.com/blog/2016/02/26/gcc-6-wmisleading-indentation-vs-goto-fail
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Trying to understand nested loops

[Dan Stromberg]

On Fri, Aug 5, 2022 at 12:30 PM GB  wrote:

> On 05/08/2022 08:56, Frank Millman wrote:
>
> > BTW, there is an indentation error in your original post - line 5 should
> > line up with line 4.
>
> As a Python beginner, I find that Python is annoyingly picky about
> indents.  And, the significance of indents is a bit of a minefield for
> beginners.
>
No, you should indent properly anyway. Python just reduces the number of
things to worry about.

Please see:
https://stromberg.dnsalias.org/~strombrg/significant-whitespace.html
...for the usual.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Trying to understand nested loops

[Dan Stromberg]

On Fri, Aug 5, 2022 at 12:35 AM  wrote:

> Hello, I’m new to learning python and I stumbled upon a question nested
> loops. This is the question below. Can you please how they arrived at 9 as
> the answer. Thanks
>
> var = 0
> for i in range(3):
>   for j in range(-2,-7,-2):
> var += 1
>  print(var)
>

A debugger is useful for more than debugging; it can also help give an
intuition for control flow.  If you single step through this snippet with a
debugger, you'll probably see what's happening.

Of if you don't have (or want) a debugger, you could change it to:

var = 0
for i in range(3):
  print('i is', i)
  for j in range(-2,-7,-2):
print('j is', j)
var += 1
print(var)

And note that 3 times 3 is 9.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dictionary order?

[Dan Stromberg]

On Mon, Aug 1, 2022 at 4:42 PM Dan Stromberg  wrote:

>
> > Yes, but I'm pretty sure that's been true for a LONG time. The hashes
>> > for small integers have been themselves for as long as I can remember.
>> > But the behaviour of the dictionary, when fed such keys, is what's
>> > changed.
>>
>> I'm not disputing either of those facts.  I'm pointing out that the
>> apparently arbitrary order of a mapping's keys becomes obvious when you
>> look at the hashes of those keys.
>>
>
> It looks like the relationship no longer holds at around keys =
> list(range(250, 260))
>
> But i == hash(i) holds for the first million values at least.
>

I could've been more clear.  int dict keys stop being stored-in-order at
near 256.

But i == hash(i) holds for the first million values, and probably more.

This suggests to me that there's something more than i == hash(i) going on
inside dict's - but it doesn't much matter what it is for my purposes.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dictionary order?

[Dan Stromberg]

On Mon, Aug 1, 2022 at 3:25 PM <2qdxy4rzwzuui...@potatochowder.com> wrote:

> On 2022-08-02 at 07:50:52 +1000,
> Chris Angelico  wrote:
>
> > On Tue, 2 Aug 2022 at 07:48, <2qdxy4rzwzuui...@potatochowder.com> wrote:
> > >
> > > On 2022-08-01 at 13:41:11 -0700,
> > > Dan Stromberg  wrote:
> > >
> > > > keys = [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7, 12, 11]
> > > >
> > > > dict_ = {}
> > > > for key in keys:
> > > > dict_[key] = 1
> > >
> > > $ python
> > > Python 3.10.5 (main, Jun  6 2022, 18:49:26) [GCC 12.1.0] on linux
> > > Type "help", "copyright", "credits" or "license" for more information.
> > > >>> [hash(x) for x in range(20)]
> > > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
> > >
> > > Just sayin'.  :-)
> >
> > Yes, but I'm pretty sure that's been true for a LONG time. The hashes
> > for small integers have been themselves for as long as I can remember.
> > But the behaviour of the dictionary, when fed such keys, is what's
> > changed.
>
> I'm not disputing either of those facts.  I'm pointing out that the
> apparently arbitrary order of a mapping's keys becomes obvious when you
> look at the hashes of those keys.
>

It looks like the relationship no longer holds at around keys =
list(range(250, 260))

But i == hash(i) holds for the first million values at least.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dictionary order?

[Dan Stromberg]

On Mon, Aug 1, 2022 at 1:41 PM Dan Stromberg  wrote:

> On 1.4 through 2.1 I got descending key order.  I expected the keys to be
> scattered, but they weren't.
>
I just noticed that 1.4 was ascending order too - so it was closer to 2.2
than 1.5.

I guess that's kind of beside the point though - it's still more ordered
than I'd've expected.
-- 
https://mail.python.org/mailman/listinfo/python-list

Dictionary order?

[Dan Stromberg]

Hi folks.

I'm still porting some code from Python 2.7 to 3.10.

As part of that, I saw a list being extended with a dict.values(), and
thought perhaps it wasn't ordered as intended on Python 2.7, even though
the problem would conceivably just disappear on 3.10.

So I decided to write a little test program to run on a variety of
CPythons, to confirm what I was thinking.

And instead I got a surprise.

On 1.4 through 2.1 I got descending key order.  I expected the keys to be
scattered, but they weren't.

On 2.2 through 3.5 I got ascending key order.  I expected the keys to be
scattered, but they weren't.

On 3.6 through 3.10 I got insertion order, as expected.

But why are 1.4 through 3.5 ordering so much?  It's like they're a treap or
red-black tree or something.  I'm pretty sure dict's used to be ordered in
a mostly-arbitrary way.

What am I missing?

Here's the little test program:

#!/usr/local/cpython-2.7/bin/python2

import sys

keys = [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7, 12, 11]

dict_ = {}
for key in keys:
dict_[key] = 1

if list(dict_.keys()) == keys:
# The order matches
print('compact')
sys.exit(0)
else:
# The order does not match
print('list(dict_): %s, keys: %s' % (list(dict_.keys()), keys))
sys.exit(1)

Here's some output (irrelevant python's deleted) when run under
https://stromberg.dnsalias.org/~strombrg/pythons/

/usr/local/cpython-1.4/bin/python (1.4) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-1.5/bin/python (1.5.2) bad  list(dict_): [15, 14, 12,
11, 10, 9, 8, 7, 6, 5, 4, 2, 1], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-1.6/bin/python (1.6.1) bad  list(dict_): [15, 14, 12,
11, 10, 9, 8, 7, 6, 5, 4, 2, 1], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.0/bin/python (2.0.1) bad  list(dict_): [15, 14, 12,
11, 10, 9, 8, 7, 6, 5, 4, 2, 1], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.1/bin/python (2.1.0) bad  list(dict_): [15, 14, 12,
11, 10, 9, 8, 7, 6, 5, 4, 2, 1], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.2/bin/python (2.2.0) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.3/bin/python (2.3.0) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.4/bin/python (2.4.0) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.5/bin/python (2.5.6) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.6/bin/python (2.6.9) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-2.7/bin/python (2.7.16) bad  list(dict_): [1, 2, 4, 5,
6, 7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-3.0/bin/python (3.0.1) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-3.1/bin/python (3.1.5) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-3.2/bin/python (3.2.5) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-3.3/bin/python (3.3.7) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-3.4/bin/python (3.4.8) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-3.5/bin/python (3.5.5) bad  list(dict_): [1, 2, 4, 5, 6,
7, 8, 9, 10, 11, 12, 14, 15], keys: [5, 10, 15, 14, 9, 4, 1, 2, 8, 6, 7,
12, 11]
/usr/local/cpython-3.6/bin/python (3.6.13) good compact
/usr/local/cpython-3.7/bin/python (3.7.0) good compact
/usr/local/cpython-3.8/bin/python (3.8.0) good compact
/usr/local/cpython-3.9/bin/python (3.9.0) good compact
/usr/local/cpython-3.10/bin/python (3.10.0) good compact

BTW, usually with pythons (the script which can be found at the URL above),
a little test program will be written to exit shell-true for success or
shell-false for failure.  But in this case I'm using the exit code not as
success+failure but as compact+notcompact.

Why are those keys so ordered?

Also, I realize that the keys could come up ordered somehow by accident,
but I tried with 30 values (not just 12), and still got the same
weirdness.  Naturally, as the number of key-value pairs goes up, the chance
of accidental ordering goes way down.

Thanks for reading!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: script folder is empty

[Dan Stromberg]

This is another reason to use:
python -m pip ...
...instead of:
   pip ...

(Or many systems want python3 -m pip)

HTH

On Sun, Jul 17, 2022 at 10:42 PM dn  wrote:

> On 18/07/2022 16.53, Scott Baer wrote:
> > I just installed Python 3.10.5 on a Windows 10 home ( Ver 21H2 OS build
> > 1904431826).
> > I'm logged in with admin privileges
> > I did a custom install with  python-3.10.5-amd64.exe to C:\Program
> > Files\Python310
> > Installed with both For all Users & PIP selected.
> > ;
> > once It was done installing, I rebooted and can run python:
> > Python 3.10.5 (tags/v3.10.5:f377153, Jun  6 2022, 16:14:13) [MSC v.1929
> 64
> > bit (AMD64)] on win32
> > Type "help", "copyright", "credits" or "license" for more information.
> 
> >
> > when I try to run:  pip --version
> > C:\Users\baerr>pip --version
> > 'pip' is not recognized as an internal or external command,
> > operable program or batch file.
> >
> > I've done some troubleshooting, and nothing is in the C:\Program
> > Files\Python310\Scripts folder.
> >
> > I"m not a noob.. but obviously, I missing something..   I doubt this is a
> > bug.
> >
> > Any help is much appreciated !!
>
> I don't use MS-Windows. Have you perused the docs?
> eg https://docs.python.org/3/using/windows.html?highlight=windows
>
> DuckDuckGo's first 'hit' for "How to Install PIP for Python on Windows"
> is https://www.liquidweb.com/kb/install-pip-windows/ - but please be
> aware that what comes-with varies according to the source used to obtain
> the copy of Python, and (perhaps) the version of MS-Win. YMMV!
> --
> Regards,
> =dn
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Extract the space group generators from Bilbao Crystallographic Server.

[Dan Stromberg]

It's good to include what you want to see as output, but it's important to
also include what you have as input.

It's also good to include what you've coded so far. It's considered good
etiquette to give it a try yourself before asking the list.

On Thu, Jul 14, 2022 at 1:03 PM hongy...@gmail.com 
wrote:

> I'm trying to extract the matrix data of "ITA-Setting F d -3 m [origin 1]"
> listed here [1], and then building an augmented matrix for each of them by
> adding the last row as "[0, 0, 0, 1]". In short, the following form is the
> ultimate-desired  result:
>
> [[[1, 0, 0, 0], [0, 1, 0, 0], [0, 0,1, 0], [0, 0, 0, 1]],
>[[-1, 0, 0, 1], [0,-1, 0, 1/2], [0, 0, 1, 1/2], [0, 0, 0, 1]],
>[[-1, 0, 0, 1/2], [0, 1, 0, 1/2], [0, 0,-1, 1], [0, 0, 0, 1]],
>[[0, 0, 1, 0], [1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 0, 1]],
>[[0, 1, 0, 3/4], [1, 0, 0, 1/4], [0, 0, -1, 3/4], [0, 0, 0, 1]],
>[[-1, 0, 0, 1/4], [0, -1, 0, 1/4], [0, 0, -1, 1/4], [0, 0, 0, 1]],
>[[1, 0, 0,  0], [0, 1, 0, 1/2], [0, 0, 1, 1/2], [0, 0, 0, 1 ]],
>[[1, 0, 0, 1/2], [0, 1, 0, 0], [0, 0, 1, 1/2], [0, 0, 0, 1]]]
>
> Any hints/tips/tricks for achieving this aim will be appreciated.
>
> [1]
> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=227=gen=a-1/8,b-1/8,c-1/8=F%20d%20-3%20m%20:1=ita
>
> Regards,
> Zhao
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subtract n months from datetime

[Dan Stromberg]

On Mon, Jun 20, 2022 at 9:45 PM Paul Bryan  wrote:

> Here's how my code does it:
>
>
> import calendar
>
> def add_months(value: date, n: int):
>   """Return a date value with n months added (or subtracted if
> negative)."""
>   year = value.year + (value.month - 1 + n) // 12
>   month = (value.month - 1 + n) % 12 + 1
>   day = min(value.day, calendar.monthrange(year, month)[1])
>   return date(year, month, day)
>

This looks interesting.

You also could add or subtract the average number of seconds in a month:
2629743.75

This has the strange property that the time of day, or even calendar day,
could change. However, it is round-trippable.
-- 
https://mail.python.org/mailman/listinfo/python-list

python.org wiki, not allowing me to log in?

[Dan Stromberg]

Hi folks.

I have a little elbow grease available, so I thought I'd edit
https://wiki.python.org/moin/BeginnersGuide/Download

...a little.

However, signing in with my google creds by clicking the little Google
button, gives me:
OpenID discovery failure, not a valid OpenID.


Does this mean I'm unauthorized to edit the page, or does it mean there's
some sort of OpenID problem?

Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Function to Print a nicely formatted Dictionary or List?

[Dan Stromberg]

On Thu, Jun 9, 2022 at 1:52 PM Michael F. Stemper 
wrote:

> On 09/06/2022 12.52, Chris Angelico wrote:
> > On Fri, 10 Jun 2022 at 03:44, Dave  wrote:
>
> >> Before I write my own I wondering if anyone knows of a function that
> will print a nicely formatted dictionary?
> >>
> >> By nicely formatted I mean not all on one line!
> >>
> >
> > https://docs.python.org/3/library/pprint.html
> >
> > from pprint import pprint
> > pprint(thing)
>
>   >>> from pprint import pprint
>   >>> d = {'two':2, 'three':5}
>   >>> pprint(d)
>   {'three': 5, 'two': 2}
>   >>>
>
> This is all on one line. That might be acceptable to the OP, but it
> doesn't actually match what he said.
>

For small outputs, pprint uses a single line.  For larger outputs, it
inserts newlines. It's intended to be human-readable more than
machine-readable.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to replace characters in a string?

[Dan Stromberg]

On Wed, Jun 8, 2022 at 1:11 AM Dave  wrote:

> I've got two that appear to be identical, but fail to compare. After
> getting the ascii encoding I see that they are indeed different, my
> question is how can I replace the \u2019m with a regular single quote mark
> (or apostrophe)?
>

Perhaps try https://pypi.org/project/Unidecode/ ?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: min, max with position

[Dan Stromberg]

On Sat, Jun 4, 2022 at 9:07 PM Greg Ewing 
wrote:

> On 5/06/22 10:07 am, dn wrote:
> > On 05/06/2022 09.50, Chris Angelico wrote:
> > min(enumerate(l), key=lambda x: x[1])
> >> (0, 1.618033)
> >
> > But, but, but which of the above characters is an 'el' and which a
> 'one'???
> > (please have pity on us old f...s and the visually-challenged!)
> >
>
> ell = l
> one = 1
> min(enumerate(ell), key=lambda x: x[one])
>
> Hope that clears it up!11!one!ell
>

I'm kind of partial to:
min((value, index) for (index, value) in enumerate(list_))

It'll return a single 2-tuple where the value of interest is at position
0.  In the event of a tie, it should give you the first such value
encountered.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Question about building Python-3.9.12 on OpenBSD 7.1

[Dan Stromberg]

It's been my understanding that there is a fundamental difference between
the *BSD's and the Linuxes.

The *BSD's have their ports system, that collects deltas against
third-party packages to build them on a *BSD.  These deltas become part of
the ports system.

The Linuxes port an application, and contribute the deltas to the package's
upstream maintainer.

For this reason, I suspect you may do well to contact the person in charge
of the port of CPython to OpenBSD.

HTH


On Thu, Jun 2, 2022 at 5:46 PM Tim Brazil  wrote:

> Hello
>
> I hope I am not breaking protocol sending this message to the list.
> This is my first posting to python-list.
>
> I am trying to build Python-3.9.12 from the ports distribution on
> on a new OpenBSD 7.1 installation.
> It is failing with the following error.
>
> Traceback (most recent call last):
>File "/usr/obj/ports/Python-3.9.12/Python-3.9.12/./setup.py", line
> 2509, in 
>  class PyBuildInstallLib(install_lib):
>File "/usr/obj/ports/Python-3.9.12/Python-3.9.12/./setup.py", line
> 2516, in PyBuildInstallLib
>  shlib_suffix = sysconfig.get_config_var("SHLIB_SUFFIX")
>File "/usr/obj/ports/Python-3.9.12/Python-3.9.12/Lib/sysconfig.py",
> line 616, in get_config_var
>  return get_config_vars().get(name)
>File "/usr/obj/ports/Python-3.9.12/Python-3.9.12/Lib/sysconfig.py",
> line 565, in get_config_vars
>  _init_posix(_CONFIG_VARS)
>File "/usr/obj/ports/Python-3.9.12/Python-3.9.12/Lib/sysconfig.py",
> line 430, in _init_posix
>  _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
> ModuleNotFoundError: No module named
> '_sysconfigdata__openbsd7_amd64-unknown-openbsd7'
> *** Error 1 in /usr/obj/ports/Python-3.9.12/Python-3.9.12 (Makefile:649
> 'sharedmods': @case "`echo X $MAKEFLAGS | sed 's/^X //;s/ -- .*//'`"...)
>
> In researching this on the internet I discovered a similar reporting on
> FreeBSD that seems to relate to regex and MULTIARZCH in the FreeBSD
> Makefile but it doesn't seem to apply to the port of my
> Makefile/environment.
>
> This is the report I am referring to:
>
>  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259896
>
> My exact problem is...
>
> I do not have z _sysconfigdata__openbsd7_amd64-unknown-openbsd7 module
> but I do have a _sysconfigdata__openbsd7_amd64-unknown-openbsd7.1.py
> module
> under:
>
> ./build/lib.openbsd-7.1-amd64-3.9/_
> sysconfigdata__openbsd7_amd64-unknown-openbsd7.1.py
>
> I suspect somewhere, it's not picking up the full 7.1 version string.
> I am having a problem figuring it out. I kindly ask if you have any
> pointers on fixing
> it. Should I log a bug or is it a OpenBSD package thing?
>
> Thanks in Advance
> Tim
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Non-deterministic set ordering

[Dan Stromberg]

On Sun, May 15, 2022 at 8:01 PM Rob Cliffe via Python-list <
python-list@python.org> wrote:

> I was shocked to discover that when repeatedly running the following
> program (condensed from a "real" program) under Python 3.8.3
>
> for p in { ('x','y'), ('y','x') }:
>  print(p)
>
> the output was sometimes
>
> ('y', 'x')
> ('x', 'y')
>
> and sometimes
>
> ('x', 'y')
> ('y', 'x')
>
> Can anyone explain why running identical code should result in
> traversing a set in a different order?
>

Sets are defined as unordered so that they can be hashed internally to give
O(1) operations for many tasks.

It wouldn't be unreasonable for sets to use a fixed-by-arbitrary ordering
for a given group of set operations, but being unpredictable deters
developers from mistakenly assuming they are ordered.

If you need order, you should use a tuple, list, or something like
https://grantjenks.com/docs/sortedcontainers/sortedset.html
-- 
https://mail.python.org/mailman/listinfo/python-list

Mypy alternatives

[Dan Stromberg]

Hello people.

I've used Mypy and liked it in combination with MonkeyType.

I've heard there are alternatives to Mypy that are faster, and I'm looking
at using something like this on a 457,000 line project.

Are there equivalents to MonkeyType that will work with these alternatives
to Mypy?

And has Mypy become the defacto standard for how type annotations should
look?  That is, are there other tools that assume Mypy's format too, and
does most doc about type annotations assume Mypy's style?

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

"py" command for Linux and Mac?

[Dan Stromberg]

Hi folks.

I heard there's a Windows-like "py" command for Linux (and Mac?).

I'm finally getting to porting a particular project's Python 2.7 code to
3.x, and one of the first steps will probably be changing a lot of "python2
script.py" to use #!/usr/bin/env python2 and chmod +x.  Then we can update
the scripts one at a time to use #!/usr/bin/env python3.

However, would this be Linux-and-Mac-only?  I'm not at all sure this code
will ever move to Windows, but in case it does, would a "py" command work
on all 3 if I use #!/usr/bin/env py?

And if so, where can I find that "py" command for Linux and Mac?

I tried searching for it in Google and on Pypi, but unsurprisingly
searching for "py" gives a buzzillion hits on other things.

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [Python-ideas] Re: New Tool Proposal

[Dan Stromberg]

On Tue, May 10, 2022 at 3:15 AM Chris Angelico  wrote:

> > It is often the case that developer write Code in Python and then
> convert to a C extension module for performance regions.
> >
> > A C extension module has a lot of boiler plate code - for instance the
> Structures required for each class, the functions for Module initialization
> etc.
> >
> > My Idea is a simple tool that uses introspection tools to take a Python
> module and to generate the relevant boiler plate for the module - including
> blank functions for the module classes and for methods. This tool would use
> type annotations (if given) to make sensible choices for parameter and
> attribute types, including using int and float directly rather than
> Internal objects (depending on tool options).
>

Two things to say about this:
1) Sometimes abandoning a pure python module for a C extension for
performance is a mistake - because Pypy is probably going to be much faster
with the pure python module
2) I've had some luck using m4 to maintain a single source file that is
used to automatically generate both pure python and cython.  This is a
little like using cpp in a C project.

For examples of #2, perhaps see:
https://stromberg.dnsalias.org/~strombrg/treap/
https://stromberg.dnsalias.org/svn/rolling_checksum_mod/trunk/
https://stromberg.dnsalias.org/~strombrg/sort-comparison/

It's often nice to keep the lines of the pure-python and cython having a
1-1 relationship, so that tracebacks report useful line numbers either
way.  However, in the treap example I've dispensed with that because some
methods were almost identical but had some boilerplate - and m4 was able to
handle that nicely at the cost of lines being 1-1.

HTH
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: tail

[Dan Stromberg]

I believe I'd do something like:

#!/usr/local/cpython-3.10/bin/python3

"""
Output the last 10 lines of a potentially-huge file.


O(n).  But technically so is scanning backward from the EOF.



It'd be faster to use a dict, but this has the advantage of working for
huge num_lines.
"""



import dbm

import os

import sys





tempfile = f'/tmp/{os.path.basename(sys.argv[0])}.{os.getpid()}'



db = dbm.open(tempfile, 'n')



num_lines = 10



for cur_lineno, line in enumerate(sys.stdin):

db[str(cur_lineno)] = line.encode('utf-8')

max_lineno = cur_lineno

str_age_out_lineno = str(cur_lineno - num_lines - 1)

if str_age_out_lineno in db:

del db[str_age_out_lineno]



for lineno in range(max_lineno, max_lineno - num_lines, -1):

str_lineno = str(lineno)

if str_lineno not in db:

break

print(db[str(lineno)].decode('utf-8'), end='')



db.close()

os.unlink(tempfile)


On Sat, Apr 23, 2022 at 11:36 AM Marco Sulla 
wrote:

> What about introducing a method for text streams that reads the lines
> from the bottom? Java has also a ReversedLinesFileReader with Apache
> Commons IO.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python/New/Learn

[Dan Stromberg]

If you already know at least one other imperative programming language:
https://wiki.python.org/moin/BeginnersGuide/Programmers

If you don't:
https://wiki.python.org/moin/BeginnersGuide/NonProgrammers


On Wed, May 4, 2022 at 7:49 PM Patrick 0511 
wrote:

> Hello, I'm completely new here and don't know anything about python. Can
> someone tell me how best to start? So what things should I learn first?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: new sorting algorithm

[Dan Stromberg]

On Mon, May 2, 2022 at 2:25 AM jan via Python-list 
wrote:

> Hi,
>
> > The median-of-three partitioning technique makes that work reasonably
> well, so it won't be pathologically slow
>
> Just to be clear because I've wondered but haven't looked into it, we
> know naive quicksorting of already-sorted data is pathalogical, but
> median-of-3 is known to fix this pathology?
>

Median-of-3 helps, but it's still possible to construct inputs that make
quicksort break down to an O(n^2) algorithm in the worst case.  These
inputs are much less common than sorting an already sorted, or
reverse-sorted list.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: new sorting algorithm

[Dan Stromberg]

On Sun, May 1, 2022 at 1:44 PM Chris Angelico  wrote:

> On Mon, 2 May 2022 at 06:43, Dan Stromberg  wrote:
> > On Sun, May 1, 2022 at 11:10 AM Chris Angelico  wrote:
> >>
> >> On Mon, 2 May 2022 at 01:53, Nas Bayedil  wrote:
> >> > We believe that using this method to develop completely new, fast
> >> > algorithms, approaching the speed of the famous *QuickSort*, the
> speed of
> >> > which cannot be surpassed, but its drawback can be circumvented, in
> the
> >> > sense of stack overflow, on some data.
> >>
> >> Hmm, actually TimSort *does* exceed the speed of quicksort for a lot
> >> of real-world data. For instance, if you take a large sorted list,
> >> append a handful of (unsorted) items to it, and then sort the list,
> >> TimSort can take advantage of the fact that the bulk of the list is
> >> sorted. It ends up significantly faster than re-sorting the entire
> >> list.
> >
> >
> > In fact, Timsort is O(n) for already-sorted data, while many quicksorts
> are O(n^2) for already-sorted data.
> >
> > Quicksort can be salvaged by using a median-of-3 partitioning, but it's
> still O(n^2) in the (less common) worst case.
> >
>
> This is true, but purely sorted data isn't a very practical case. The
> case of mostly-sorted data IS practical, though, so it's a quite big
> win that it can be close to O(n), and still faster than inserting each
> item individually.
>

You seem to be of the impression that nearly-sorted data isn't an uphill
battle with a straightforward quicksort.

I'm having a hard time convincing myself of that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: tail

[Dan Stromberg]

On Sun, May 1, 2022 at 3:19 PM Cameron Simpson  wrote:

> On 01May2022 18:55, Marco Sulla  wrote:
> >Something like this is OK?
>

Scanning backward for a byte == 10 in ASCII or ISO-8859 seems fine.

But what about Unicode?  Are all 10 bytes newlines in Unicode encodings?

If not, and you have a huge file to reverse, it might be better to use a
temporary file.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: new sorting algorithm

[Dan Stromberg]

This probably should start out as a module on Pypi.

Is the sorting stable? Python guarantees that.

On Sun, May 1, 2022 at 8:53 AM Nas Bayedil  wrote:

> *Dear, Sir/Madam*
>
>
> Let me first tell you briefly who we are and where we are from, what we do.
>
> My name is Nas (full name Nasipa Bayedil) from Kazakhstan.
>
> In December 2020, we registered a company online in Dover, Delaware, the
> United States, because all major corporations are based in the United
> States. Direction of the company: research and development technologies of
> sorting and searching in databases and in Big Data. Our research and
> developments should be of interest to a rapidly growing tech market.
>
> My father Nurgali has a mathematics education, he has been interested in
> mathematics' and physics all his life, now he is retired and continues to
> study his favorite mathematics, when he became a pensioner to this love an
> interest in programming was added. And this new interest in programming led
> him to invent his own method of developing sorting algorithms.
>
> *In a nutshell about what we are doing.*
>
> We have developed several variants of sorting algorithms based on new ideas
> and improvements to well-known ideas.
>
> Our development is carried out in the application of the idea of the
> article Peter McIlroy's 1993 paper "Optimistic Sorting and Information
> Theoretic Complexity", which was implemented by *Tim Peters TimSort in a
> sorting algorithm in 2002 for use in Python.*
>
> The variant implemented by Tim Peters is improved by us as it does not take
> full advantage of Peter McIlroy's idea.
>
> This has been achieved through the development of new methods:
>
>   1. data recognition
>
>   2. data fusion
>
>
>
> We also used the research work Sorting N-Elements Using Natural Order: A
> New Adaptive Sorting Approach** but added some new ideas of our own. As a
> result, we got a hybrid NDsort algorithm, which uses the above particular
> algorithms.
>
>
>
> Existing sorting algorithms that are used in practice, very fast, I will
> not list their advantages; for improvement let's just indicate that some of
> them start to work slowly on certain types of data. The author tried to
> deal with this from a general, *mathematical position*, and not from the
> point of view of programming techniques, *and achieved the goal*.
>
> Fragments of the general theory of sorting by comparison have been
> developed, *several facts have been discovered that are subject to
> patenting *and which allow developing various variants of sorting
> algorithms.
>
> We believe that using this method to develop completely new, fast
> algorithms, approaching the speed of the famous *QuickSort*, the speed of
> which cannot be surpassed, but its drawback can be circumvented, in the
> sense of stack overflow, on some data.
>
> Additionally our general sorting algorithm can be applied to design data
> sorting chips. This is, of course, the business of specialists in this
> field, but we believe that our algorithm opens up a new approach to this
> problem. The fact is that there are currently two ways to develop data
> sorting chips. The first is that data is entered in parallel; the second is
> that data is entered sequentially. Our sorting algorithm allows
> parallel-sequential data input, which would speed up the entire sorting
> process.
>
> I hope our research will pique your interest. The distinctive features of
> our sorting algorithms are the minimization of the number of comparisons, a
> new approach to the sorting problem, the maximum use of common sense, and
> the use of a principle from philosophy "Occam's razor".
>
> We use C# (Visual Studio 2019 demo version) for research and write
> fragments of algorithms. We would like to ask for help from Python.org
> engineers in implementation of our idea for the Python language together.
>
> We will be glad to hear from you any comments and questions about our idea.
>
>
>
>
>
> Kind regards,
>
> Nas Bayedil
>
>
>
> Twitter: @NasBayedil
>
> Website: www.bazony.com.kz
>
>
>
> **June 2010 Journal of Computer Science 6(2) Project: Algorithm Analysis,
>
> Authors: Shamim Akhter, International University of Business Agriculture
> and Technology M. Tanveer Hasan.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: logging library python

[Dan Stromberg]

You probably want getLogger(__name__)

...or something close to it.

‪On Wed, Apr 27, 2022 at 12:58 PM ‫תמר ווסה‬‎  wrote:‬

> hello,
> we have many scripts of one project. what is the right way to define the
> logger to all scripts? we tried to create class that define the logger
> configuration+generic function (handler functions to write generic log
> message),  and each script define an instance of the class. But the log
> info is reference to the class script and not the right script. Should we
> need not to use the class ? should we need define logger to each script ?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Style for docstring

[Dan Stromberg]

On Fri, Apr 22, 2022 at 12:56 PM Michael F. Stemper <
michael.stem...@gmail.com> wrote:

> I'm writing a function that is nearly self-documenting by its name,
> but still want to give it a docstring. Which of these would be
> best from a stylistic point of view:
>
>
>Tells caller whether or not a permutation is even.
>
>Determines if a permutation is even. (Alternative is that it's odd.)
>
>Returns True if permutation is even, False if it is odd.
>
>
> (Before somebody suggests it, I'm not going to put six weeks' worth
> of a course in group theory in there to help somebody who doesn't
> know what those standard terms mean.)
>

Maybe try pydocstyle?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why no list as dict key?

[Dan Stromberg]

On Wed, Apr 20, 2022 at 7:23 PM Abdur-Rahmaan Janhangeer <
arj.pyt...@gmail.com> wrote:

> Maybe hashes should point to an object rather than being the hash of an
> object themselves.
> Maybe the speed drop is not worth it.
>

If you need mutable keys, you /might/ create a dict-like-object using what
functional languages call an association list.  But these have O(n)
lookups, not O(1) like a dict (AKA hash table).  And even then, if you have
keys a and b, a != b, and you modify b to look just like a, you could get a
bit of a mess from the two equal keys mapping to two different values.

It is possible to have a dict that allows duplicates though.  Usually
that's not what you want, but sometimes it is, EG:
https://stromberg.dnsalias.org/~strombrg/dupdict_mod/

Trees have much the same issue as a dict (hash table) in this case.  These
are O(logn) or so.

The problem is hash tables and trees don't want keys changing "out from
under them".  It causes things they're assuming are invariant, to vary.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python app on a Mac

[Dan Stromberg]

On Fri, Apr 15, 2022 at 11:43 AM Alan Gauld  wrote:

> I've just migrated from a Linux PC to a Mac mini running Monterey.
>
I'm using a Mac for work lately.  I miss Linux.  I feel like MacOS isn't
nearly as good at multimonitor setups as Cinnamon.

Does anyone know how to launch a Python program from the
> desktop without a Terminal window (or at least with an
> iconified one!) Does it require Applescript or somesuch?
>

You could try appify:
https://stromberg.dnsalias.org/~strombrg/mactools/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Comparing sequences with range objects

[Dan Stromberg]

Two more things:
1) There are also modules that do interval arithmetic with reals, not just
integers.  EG: https://pyinterval.readthedocs.io/en/latest/
2) If you don't mind a small amount of inaccuracy you can often get things
down to less than one bit per element for presence/absence, using a bloom
filter.  EG: https://pypi.org/project/drs-bloom-filter/

On Sun, Apr 10, 2022 at 5:31 PM Dan Stromberg  wrote:

>
> It sounds a little like you're looking for interval arithmetic.
>
> Maybe https://pypi.org/project/python-intervals/1.5.3/ ?
>
> On Thu, Apr 7, 2022 at 4:19 AM Antoon Pardon  wrote:
>
>> I am working with a list of data from which I have to weed out duplicates.
>> At the moment I keep for each entry a container with the other entries
>> that are still possible duplicates.
>>
>> The problem is sometimes that is all the rest. I thought to use a range
>> object for these cases. Unfortunatly I sometimes want to sort things
>> and a range object is not comparable with a list or a tuple.
>>
>> So I have a list of items where each item is itself a list or range
>> object.
>> I of course could sort this by using list as a key function but that
>> would defeat the purpose of using range objects for these cases.
>>
>> So what would be a relatively easy way to get the same result without
>> wasting
>> too much memory on entries that haven't any weeding done on them.
>>
>> --
>> Antoon Pardon.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Comparing sequences with range objects

[Dan Stromberg]

It sounds a little like you're looking for interval arithmetic.

Maybe https://pypi.org/project/python-intervals/1.5.3/ ?

On Thu, Apr 7, 2022 at 4:19 AM Antoon Pardon  wrote:

> I am working with a list of data from which I have to weed out duplicates.
> At the moment I keep for each entry a container with the other entries
> that are still possible duplicates.
>
> The problem is sometimes that is all the rest. I thought to use a range
> object for these cases. Unfortunatly I sometimes want to sort things
> and a range object is not comparable with a list or a tuple.
>
> So I have a list of items where each item is itself a list or range object.
> I of course could sort this by using list as a key function but that
> would defeat the purpose of using range objects for these cases.
>
> So what would be a relatively easy way to get the same result without
> wasting
> too much memory on entries that haven't any weeding done on them.
>
> --
> Antoon Pardon.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Cython compiler is 20 years old today !

[Dan Stromberg]

On Mon, Apr 4, 2022 at 7:42 AM Stefan Behnel  wrote:

> Dear Python community,
>
> it's now 20 years since Greg Ewing posted his first announcement of Pyrex,
> the tool that is now known and used under the name Cython.
>
> https://mail.python.org/pipermail/python-list/2002-April/126661.html
>

That's a cool anniversary.

I often shake my head in wonder when I see people still writing extension
modules in C instead of Cython.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Sharing part of a function

[Dan Stromberg]

On Sun, Apr 3, 2022 at 2:46 PM Cecil Westerhof via Python-list <
python-list@python.org> wrote:

> Betty Hollinshead  writes:
>
> > "Memoising" is the answer -- see "Python Algorithms" by Magnus Lie
> Hetland.
> > In the mean time, here a simplified version of "memoising" using a dict.
> > This version handles pretty large fibonacci numbers!
> >
> > # fibonacci sequence
> > # memoised - but using a simple dictionary (see Python Algorithms, p177)
> >
> > memo= {}
> > memo[1] = 1
> > memo[2] = 2
> >
> > def fib(n):
> > if n in memo:
> > return memo[n]
> > else:
> > memo[n] = fib(n - 1) + fib(n - 2)
> > return memo[n]
> >
> >
> > print(100, fib(100))
> > print(memo)
>
> No, it is not. It is the answer on a completely different question.
> Nice, but not what I was asking about. And there is even an error in
> the solution.
>
> By the way: it is better to do it iterative. Try (when not done a
> calculation before) fib(3_000).
>

I think I missed part of this conversation, but here is how I've done
fibonacci numbers in the past, using functools.lru_cache:

#!/usr/local/cpython-3.8/bin/python

"""Compute the first n numbers in the fibonacci sequence."""



import functools

import sys





@functools.lru_cache(maxsize=None)  # pylint: disable=no-member

def find_recursive(number):

"""Find a Fibonacci number recursively - without the callstack
explosion."""
assert number >= 0

if number == 0:

return 0

if number == 1:

return 1

result = find_recursive(number - 1) + find_recursive(number - 2)

return result





def main():

"""Compute fibonacci numbers."""

top = 5000

if sys.argv[1:]:

top = int(sys.argv[1])

if sys.argv[2:]:

sys.stderr.write('Usage: {} 5000\n'.format(sys.argv[0]))

sys.exit(1)

for number in range(1, top + 1):

print(number, find_recursive(number))





main()
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compiling and Linking pre-built Windows Python libraries with C++ files on Linux for Windows

[Dan Stromberg]

On Fri, Mar 18, 2022 at 8:03 PM Ankit Agarwal  wrote:

> Hi,
>
> This is a very specific question. I am trying to figure out whether or not
> I can use pre-built python libraries and headers on Windows in a MinGW
> build on Linux. Essentially I have some python and C++ code which interface
> via cython and pybind. I want to build a self contained C++ binary for
> Windows with the MinGW compiler that runs on Linux. For both Cython and
> PyBind, they need to compile with the python headers and link against the
> python DLL in order to run on Windows.
>
> I know that the python DLL specifically are compiled with the MSVC
> compiler, however since it is in C, the ABI between the DLL should be
> compatible with MinGW, and I should be able to import and link against it.
> My question is will this work, or will there be some other problem that I
> might run into.
>
>
I haven't tried this.

However, I used to cross-compile the Linux kernel from Solaris on Sparc to
Intel.  I just had to:
1) Get the relevant headers and libraries on Solaris
2) Deal with the byte sex issues - Sparc is Big Endian, Intel is Little
Endian
3) Natively compile a little bit of code that was needed by the build
process

You appear to be aware of #1.

You probably won't need to worry about #2, since you're going Intel ->
Intel.

#3 could be an issue for you, but it's just a matter of using two different
compilers for some different parts of the build process - one native, one
cross.

I'd try a little hello world first, then worry about your larger project.

You could also put some feelers out about Cython and Pybind too, to see if
they've been used for cross-compilation before.  If yes, you're probably in
like Flynn, otherwise it could potentially turn out to be a big project.

If cross-compilation doesn't work out, you could probably set up a Windows
virtual machine with an sshd, and build on that.

Either way, you may find Wine useful for testing.

HTH.
-- 
https://mail.python.org/mailman/listinfo/python-list

Weird strace of #! python script

[Dan Stromberg]

Hi folks.

First off, I know, python2 is ancient history.  Porting to 3.x is on my
list of things to do (though I'm afraid it's not yet at the top of the
list), and the same thing happens with python3.

So anyway, I'm strace'ing a #!/usr/bin/python2 script.

I expected to see an exec of /usr/bin/python2, but I don't.  I just see an
exec of /tmp/t.

As follows:
tact@celery_worker:/app$ strace -f -s 1024 -o /tmp/t.strace /tmp/t
^Z
[1]+  Stopped strace -f -s 1024 -o /tmp/t.strace /tmp/t
tact@celery_worker:/app$ bg
[1]+ strace -f -s 1024 -o /tmp/t.strace /tmp/t &
tact@celery_worker:/app$ ps axf
  PID TTY  STAT   TIME COMMAND
 1163 pts/0Ss 0:00 bash
 1363 pts/0S  0:00  \_ strace -f -s 1024 -o /tmp/t.strace /tmp/t
 1366 pts/0S  0:00  |   \_ /usr/bin/python2 /tmp/t
 1367 pts/0R+ 0:00  \_ ps axf
tact@celery_worker:/app$ fg
bash: fg: job has terminated
[1]+  Donestrace -f -s 1024 -o /tmp/t.strace /tmp/t
tact@celery_worker:/app$ grep execve /tmp/t.strace
1366  execve("/tmp/t", ["/tmp/t"], 0x7ffd89f9c3b8 /* 49 vars */) = 0
tact@celery_worker:/app$

I've deleted some irrelevant processes from the 'ps axf'.

/tmp/t is actually just:
tact@celery_worker:/app$ cat /tmp/t
#!/usr/bin/python2

import time

time.sleep(10)


Was this some sort of security feature I never heard about?  I'm tracing a
very simple time.sleep(10) here, but the same thing is (not) happening in a
larger script that I need to track down a bug in.

Is there a way I can coax Linux and/or strace to show all the exec's, like
they used to?  Not having them makes me wonder what else is missing from
the strace report.

I'm on a Debian 11.2 system with strace 5.10 and Python 2.7.18.

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Suggestion for Linux Distro (from PSA: Linux vulnerability)

[Dan Stromberg]

On Thu, Mar 10, 2022 at 5:04 AM Marco Sulla 
wrote:

> On Thu, 10 Mar 2022 at 04:50, Michael Torrie  wrote:
> >
> > On 3/9/22 13:05, Marco Sulla wrote:
> > > So my laziness pays. I use only LTS distros, and I update only when
> > > there are security updates.
> > > PS: any suggestions for a new LTS distro? My Lubuntu is reaching its
> > > end-of-life. I prefer lightweight debian-like distros.
> >
> > Maybe Debian itself?
>
> I tried Debian on a VM, but I found it too much basical. A little
> example: it does not have the shortcut ctrl+alt+t to open a terminal
> that Ubuntu has. I'm quite sure it's simple to add, but I'm starting
> to be old and lazy...
>
That's an attribute of your desktop environment, not the Linux distribution.

EG: I'm using Debian with Cinnamon, which does support ctrl-alt-t.

Some folks say the desktop environment matters more than the distribution,
when choosing what OS to install.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Sun, Jan 23, 2022 at 9:59 AM Dan Stromberg  wrote:

>
> Hi folks.
>
> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
> Linux and macOS 11.5.  It's at
> https://stromberg.dnsalias.org/~strombrg/hcm/ if you're curious.
>
> It works the way I want on Linux, but on macOS I seem to have to start it
> from the command line, like:
> hcm --gui
> ...because I don't know how to create a macOS "app" that goes under
> /Applications.
>
> I don't really care about having a single executable on macOS, and I don't
> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
> running "make install" and putting a #!'d script under /Applications with
> appropriate metadata - but if it's easier to do a single executable, .dmg
> or .pkg, I'd be fine with that.
>

It turns out that using something called "appify" does this well.

I found it on a gist I believe.

I liked it so much that I adopted it and put it at
https://stromberg.dnsalias.org/~strombrg/mactools/ , with a few style
changes and small bug fixes.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python

[Dan Stromberg]

Whatever happened to sending a URL to a specific answer in a FAQ list?

On Thu, Mar 3, 2022 at 12:52 PM Dan Ciprus (dciprus) via Python-list <
python-list@python.org> wrote:

> if OP formulates question the way he/she did, it's not worth to respond to
> it.
> There is plenty of similar questions in the archive.
>
> On Tue, Feb 22, 2022 at 07:07:54AM -0700, Mats Wichmann wrote:
> >On 2/21/22 23:17, SASI KANTH REDDY GUJJULA wrote:
> >> Pip files are not installing after the python 3.10.2 version installing
> in my devise. Please solve this for me.
> >
> >Please ask a clearer question.
> >
> >Can you tell us what "are not installing" means? Are you getting
> >permission errors?  Are you installing and then unable to import what
> >you have installed?  Something else?
> >
> >--
> >https://mail.python.org/mailman/listinfo/python-list
>
> --
> Daniel Ciprus  .:|:.:|:.
> CONSULTING ENGINEER.CUSTOMER DELIVERY   Cisco Systems Inc.
>
> [ curl -L http://git.io/unix ]
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Behavior of the for-else construct

[Dan Stromberg]

On Thu, Mar 3, 2022 at 5:24 AM computermaster360 <
computermaster...@gmail.com> wrote:

> I want to make a little survey here.
>
> Do you find the for-else construct useful? Have you used it in
> practice? Do you even know how it works, or that there is such a thing
> in Python?
>
> I have used it maybe once. My issue with this construct is that
> calling the second block `else` doesn't make sense; a much more
> sensible name would be `then`.
>

I use it and like it.

You need break's to make it useful - then it cuts down on unnecessary
booleans.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: One-liner to merge lists?

[Dan Stromberg]

On Fri, Feb 25, 2022 at 3:15 PM Chris Angelico  wrote:

> But ultimately, that's still the same as sum(), and it's no more
> readable than chain(), so I'd still be inclined to go with chain and
> then wrap it in a function if you need the clarity.
>

"Need" the clarity?  Please tell me you don't think clarity is for the
feeble minded.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Dan Stromberg]

On Fri, Feb 25, 2022 at 8:12 AM BELAHCENE Abdelkader <
abdelkader.belahc...@enst.dz> wrote:

> Hi,
> a lot of people think that C (or C++) is faster than python, yes I agree,
> but I think that's not the case with numpy, I believe numpy is faster than
> C, at least in some cases.
>

This is all "last time I heard".

numpy is written, in significant part, in Fortran.

Fortran, especially for matrix math with variable dimensions, can be faster
than C.

Fortran, (still last I heard) did not support pointers, which gives Fortran
compilers the chance to exploit a very nice class of optimizations you
can't use nearly as well in languages with pointers.

I used to code C to be built with the "noalias" optimization, to get much
of the speed of Fortran in C.  But it required using an error prone subset
of C without good error detection.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Dan Stromberg]

On Fri, Feb 25, 2022 at 9:03 PM Chris Angelico  wrote:

> On Sat, 26 Feb 2022 at 15:39, Avi Gross via Python-list
>  wrote:
> > Take interpreted languages including Python and R that specify all kinds
> of functions that may be written within the language at first. Someone may
> implement a function like sum() (just an example) that looks like the sum
> of a long list of items is the first item added to a slightly longer sum of
> the remaining items. It stops when the final recursive sum is about to be
> called with no remaining arguments. Clearly this implementation may be a
> tad slow. But does Python require this version of sum() or will it allow
> any version that can be called the same way and returns the same results
> every time?
> >
>
> That's also true of C and pretty much every language I know of. They
> define semantics, not implementation.
>

This comes back to something we've discussed before.

A language that is described primarily by a reference implementation rather
than a standard, runs the risk of being defined by that implementation.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: One-liner to merge lists?

[Dan Stromberg]

On Tue, Feb 22, 2022 at 7:46 AM David Raymond 
wrote:

> > Is there a simpler way?
>
> >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
> >>> [a for b in d.values() for a in b]
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>>
>

I like that way best.

But I'd still:
1) use a little more descriptive identifiers
2) put it in a function with a descriptive name
3) give the function a decent docstring
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why does not Python accept functions with no names?

[Dan Stromberg]

On Sun, Feb 20, 2022 at 7:33 AM Abdur-Rahmaan Janhangeer <
arj.pyt...@gmail.com> wrote:

> Greetings list.
>
> Out of curiosity, why doesn't Python accept
> def ():
> return '---'
>
> ()
>
> Where the function name is ''?
>

() is already an empty tuple.  It would break code to change this.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Long running process - how to speed up?

[Dan Stromberg]

On Sat, Feb 19, 2022 at 3:29 AM Shaozhong SHI 
wrote:

> I have a cvs file of 932956 row and have to have time.sleep in a Python
> script.  It takes a long time to process.
>
> How can I speed up the processing?  Can I do multi-processing?
>

How are you doing it right now?

Are you using the csv module?

You might be able to use the GNU "split" command as a prelude to using the
csv module in combination with multiprocessing.  GNU split comes with
Linuxes, but I'm sure you can get it for Windows.  MacOS comes with a
rather less powerful "split" command, but it still might work for you.

You also could try Pypy3.

HTH.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pypy with Cython

[Dan Stromberg]

The best answer to "is this slower on Pypy" is probably to measure.

Sometimes it makes sense to rewrite C extension modules in pure python for
pypy.

On Thu, Feb 3, 2022 at 7:33 AM Albert-Jan Roskam 
wrote:

>Hi,
>I inherited a fairly large codebase that I need to port to Python 3.
> Since
>the program was running quite slow I am also running the unittests
> against
>pypy3.8. It's a long running program that does lots of pairwise
>comparisons of string values in two files. Some parts of the program
> (e.g
>a modulo 11 digit check) are implemented in Cython. Should I use pure
>Python instead when using Pypy? I compiled the Cython modules for pypy
> and
>they work, but I'm afraid they might just slow things down.
>Thanks!
>Albert-Jan
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Wed, Jan 26, 2022 at 2:35 PM Barry  wrote:

>
>
> On 26 Jan 2022, at 05:17, Dan Stromberg  wrote:
>
>
> On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg  wrote:
>
>>
>> On Tue, Jan 25, 2022 at 2:23 PM Barry  wrote:
>>
>>>
>>> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
>>>
>>> 
>>>
>>> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
>>>
>>>>
>>>> I do not have experience with great, but you might try pyinstaller.
>>>> I use it to make a PyQt Mac app successfully.
>>>>
>>>> It’s command line plus setup script.
>>>>
>>>
>>> I wound up doing:
>>> 1) pyinstaller, as normal, but this created a broken all-encompassing
>>> binary of my script.  At least it gave me the metadata I needed though.
>>>
>>>
>>> You mean it created a .app bundle?
>>>
>>> That is the way that macOS makes it trivia to install apps
>>> Just by drag and drop in /Applications.
>>>
>>
>> Yes, it created an hcm.app for me.  But the executable it created didn't
>> work.  Hence the hack.
>>
>
> $ file /Applications/hcm.app/Contents/MacOS/hcm
> cmd output started 2022 Tue Jan 25 09:00:54 PM PST
> /Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64
>
>
> It’s intended to be started as an app.
>
> What if you double click the app? Does it work?
>
No, it does not start that way either.


> Also you can use the open command to run use the app name you give it.
>
 Thanks for the tip.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg  wrote:

>
> On Tue, Jan 25, 2022 at 2:23 PM Barry  wrote:
>
>>
>> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
>>
>> 
>>
>> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
>>
>>>
>>> I do not have experience with great, but you might try pyinstaller.
>>> I use it to make a PyQt Mac app successfully.
>>>
>>> It’s command line plus setup script.
>>>
>>
>> I wound up doing:
>> 1) pyinstaller, as normal, but this created a broken all-encompassing
>> binary of my script.  At least it gave me the metadata I needed though.
>>
>>
>> You mean it created a .app bundle?
>>
>> That is the way that macOS makes it trivia to install apps
>> Just by drag and drop in /Applications.
>>
>
> Yes, it created an hcm.app for me.  But the executable it created didn't
> work.  Hence the hack.
>

More specifically:
$ /Applications/hcm.app/Contents/MacOS/hcm --gui
cmd output started 2022 Tue Jan 25 09:00:33 PM PST
Traceback (most recent call last):
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 146, in

_run()
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 129, in
_run
exec(compile(source, path, "exec"), globals(), globals())
  File "/Applications/hcm.app/Contents/Resources/hcm.py", line 1950, in

import gi
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 664, in _load_unlocked
  File "", line 627, in
_load_backward_compatible
  File "", line 259, in load_module
  File "gi/__init__.pyc", line 40, in 
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 664, in _load_unlocked
  File "", line 627, in
_load_backward_compatible
  File "", line 259, in load_module
  File "gi/_gi.pyc", line 14, in 
  File "gi/_gi.pyc", line 10, in __load
  File "imp.pyc", line 342, in load_dynamic
  File "", line 1007, in _find_and_load
  File "", line 986, in _find_and_load_unlocked
  File "", line 664, in _load_unlocked
  File "", line 627, in
_load_backward_compatible
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 36, in
load_module
return imp.load_module(
  File "imp.pyc", line 244, in load_module
  File "imp.pyc", line 216, in load_package
  File "", line 710, in _load
AttributeError: 'NoneType' object has no attribute 'name'
2022-01-25 21:00:34.576 hcm[62695:1322031] hcm Error
^C^\Quit: 3
above cmd output done2022 Tue Jan 25 09:00:42 PM PST
dstromberg@Daniels-Mini:~/src/home-svn/hcm/trunk x86_64-apple-darwin20.6.0
61933

$ file /Applications/hcm.app/Contents/MacOS/hcm
cmd output started 2022 Tue Jan 25 09:00:54 PM PST
/Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64

But if I replace /Applications/hcm.app/Contents/MacOS/hcm with a symlink to
a wrapper shell script, hcm runs fine from the Applications menu.

It seems that gi.repository.Gtk applications are not packaged correctly by
pyinstaller and py2app.  Some Python modules require a little assistance to
be packaged up into a Mach-O executable neatly by such tools.  But it's
just not that important to me to have a Mach-O of my app.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Tue, Jan 25, 2022 at 2:23 PM Barry  wrote:

>
> On 25 Jan 2022, at 02:56, Dan Stromberg  wrote:
>
> 
>
> On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:
>
>>
>> I do not have experience with great, but you might try pyinstaller.
>> I use it to make a PyQt Mac app successfully.
>>
>> It’s command line plus setup script.
>>
>
> I wound up doing:
> 1) pyinstaller, as normal, but this created a broken all-encompassing
> binary of my script.  At least it gave me the metadata I needed though.
>
>
> You mean it created a .app bundle?
>
> That is the way that macOS makes it trivia to install apps
> Just by drag and drop in /Applications.
>

Yes, it created an hcm.app for me.  But the executable it created didn't
work.  Hence the hack.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: mac app from a python script?

[Dan Stromberg]

On Sun, Jan 23, 2022 at 1:37 PM Barry  wrote:

>
> I do not have experience with great, but you might try pyinstaller.
> I use it to make a PyQt Mac app successfully.
>
> It’s command line plus setup script.
>

I wound up doing:
1) pyinstaller, as normal, but this created a broken all-encompassing
binary of my script.  At least it gave me the metadata I needed though.
2) overwriting /Applications/hcm.app/Contents/MacOS/hcm with a proper
#!/usr/bin/env python3 script

This mostly works.  It's a kinda ugly hack, and it doesn't stay in the dock
after starting it.

There should be a way of installing a python GUI in the macOS Applications
list, without having to bundle everything up into a big binary.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: "undefined symbol" in C extension module

[Dan Stromberg]

On Sun, Jan 23, 2022 at 9:02 AM Robert Latest via Python-list <
python-list@python.org> wrote:

> Dan Stromberg wrote:
> > Perhaps try:
> > https://stromberg.dnsalias.org/svn/find-sym/trunk
> >
> > It tries to find symbols in C libraries.
> >
> > In this case, I believe you'll find it in -lpythonx.ym
>
> Thanks! Found out that ldd produces many errors also with working python
> libraries. Turns out I tried to revive a package from v2.7. Need to adapt
> it to 3.x
>

I wonder if you're missing a -Wl,-rpath -Wl,/path/to/library/dir
?
-- 
https://mail.python.org/mailman/listinfo/python-list

mac app from a python script?

[Dan Stromberg]

Hi folks.

I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
if you're curious.

It works the way I want on Linux, but on macOS I seem to have to start it
from the command line, like:
hcm --gui
...because I don't know how to create a macOS "app" that goes under
/Applications.

I don't really care about having a single executable on macOS, and I don't
really care about creating a .dmg or .pkg file. I'd be perfectly happy just
running "make install" and putting a #!'d script under /Applications with
appropriate metadata - but if it's easier to do a single executable, .dmg
or .pkg, I'd be fine with that.

I've experimented with a few different options for this (months ago),
mostly py2app, but it doesn't appear to like gi.repository.Gtk much.

What's the most straightforward way of installing a Python script under
/Applications on macOS?  I'd -love- to find a way of doing something
analogous to Linux' desktop-file-install - that is, something that isn't
click-happy, but GUI's are acceptable too if they'll get the job done.

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: "undefined symbol" in C extension module

[Dan Stromberg]

Perhaps try:
https://stromberg.dnsalias.org/svn/find-sym/trunk

It tries to find symbols in C libraries.

In this case, I believe you'll find it in -lpythonx.ym


On Sat, Jan 22, 2022 at 12:43 PM Robert Latest via Python-list <
python-list@python.org> wrote:

> Hi guys,
>
> I've written some CPython extension modules in the past without problems.
> Now
> after moving to a new Archlinux box with Python3.10 installed, I can't
> build
> them any more. Or rather, I can build them but not use them due to
> "undefined
> symbols" during linking. Here's ldd's output when used on the "spam"
> example
> library from the docs:
>
> linux-vdso.so.1 (0x7ffe2454a000)
> libc.so.6 => /usr/lib/libc.so.6 (0x7fb6b6eb9000)
> /usr/lib64/ld-linux-x86-64.so.2 (0x7fb6b70a4000)
> undefined symbol: PyObject_Init (./build/lib.linux-x86_64-3.10/
> spam.cpython-310-x86_64-linux-gnu.so)
>
> [and more of the same]
>
> It seems that I need to install some library to make this work, but which
> one?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A Newspaper for Python Mailing Lists

[Dan Stromberg]

There's also Python Weekly:
https://www.pythonweekly.com/

On Sat, Jan 8, 2022 at 10:29 PM Abdur-Rahmaan Janhangeer <
arj.pyt...@gmail.com> wrote:

> Well yes XD though LWN covers Py topics well when it wants
>
>
> 1. Yes sure, did not expect RSS interest
> 2. Excuse my blunder, will do!
>
>

> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Starting using Python

[Dan Stromberg]

On Mon, Jan 3, 2022 at 8:06 AM Joao Marques  wrote:

> Good morning: I have a very simple question: I want to start writing
> programs in Python so I went to the Microsoft Store and installed
> Python3.9. No problem so far. I would prefer to have a gui interface, an
> interface that I can use file-->Open and File-->Save as, as I see it on
> different videos. How can I get it? Because my problem is to run the
> programs I have already written and saved on a *.py file in my own working
> directory, not in the Python's CWD directory.
> Can you please help?
> I am running Windows 10 Pro version 20H2
>

I'm thinking you probably want PyCharm or VSCode.  Give them each a google
search and compare a little.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PYTHON NOT WORKING

[Dan Stromberg]

On Sat, Jan 1, 2022 at 1:52 PM the derek team  wrote:

> HI, I am trying to use python 3.10-1 on windows but, When I try to open
> python, it crashes. Anaconda also does not work. When I try to use the
> powershell, it gives me an error message saying that this is not recognized
> as a valid cmdlet. Please help.
>

Please describe where you obtained your CPython.

Also, please cut-and-paste the full text of the errors you're receiving.

HTH.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Call julia from Python: which package?

[Dan Stromberg]

Last I heard, Pypy itself will remain written in a statically typed dialect
of Python that is closest to Python 2.7.  However, what's written in that
language includes JIT-compiled interpreters for both Python 2.x and Python
3.x.

On Tue, Dec 21, 2021 at 9:15 AM Albert-Jan Roskam 
wrote:

>Hi all,
>Thank you very much for your valuable replies! I will definitely do some
>tracing to see where the bottlenecks really are. It's good to know that
>pypy is still alive and kicking, I thought it was stuck in py2.7. I will
>also write a mini program during the holiday to see how this
> Julia/Python
>interaction might work. The little bit of experience with Julia more or
>less coincides with what Oscar mentioned: a lot of "warm up" time. This
> is
>actually a py2.7 project that I inherited. I was asked to convert it to
>py3.8.
>Thanks and merry xmas!
>Albert-Jan
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Call julia from Python: which package?

[Dan Stromberg]

On Fri, Dec 17, 2021 at 7:02 AM Albert-Jan Roskam 
wrote:

> Hi,
>
> I have a Python program that uses Tkinter for its GUI. It's rather slow so
> I hope to replace many or all of the non-GUI parts by Julia code. Has
> anybody experience with this? Any packages you can recommend? I found three
> alternatives:
>
> * https://pyjulia.readthedocs.io/en/latest/usage.html#
> * https://pypi.org/project/juliacall/
> * https://github.com/JuliaPy/PyCall.jl
>
> Thanks in advance!
>

I have zero Julia experience.

I thought I would share this though:
https://stromberg.dnsalias.org/~strombrg/speeding-python/

Even if you go the Julia route, it's probably still best to profile your
Python code to identify the slow ("hot") spots, and rewrite only them.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Subprocess Connection error

[Dan Stromberg]

Hi.

It's important to include the full text of whatever error messages you're
getting.

If you can, you should also include your code, or better: include an
http://sscce.org/

Can you start the python interpreter from bash, powershell or cmd.exe? If
not, please again include the full text of any error messages you get.

BTW, it's important to cut and paste TEXT - don't just send screenshots.

Good luck.

On Tue, Nov 23, 2021 at 8:03 AM  wrote:

>Hello,
>
>
>
>I have been 3.8.10 version of python until I ran the code for tkinter
> and
>I started getting subprocess connection error. After this I haven’t been
>able to open python for use as I get this error upon startup.
>
>
>
>Please help resolve the matter soon.
>
>
>
>Best regards,
>
>Nuha
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pytest segfault, not with -v

[Dan Stromberg]

On Sat, Nov 20, 2021 at 10:59 AM Dan Stromberg  wrote:

>
>
> On Sat, Nov 20, 2021 at 10:09 AM Marco Sulla 
> wrote:
>
>> I know how to check the refcounts, but I don't know how to check the
>> memory usage, since it's not a program, it's a simple library. Is
>> there not a way to check inside Python the memory usage? I have to use
>> a bash script (I'm on Linux)?
>>
>
> ps auxww
> ...can show you how much memory is in use for the entire process.
>
> It's commonly combined with grep, like:
> ps auxww | head -1
> ps auxww | grep my-program-name
>
> Have a look at the %MEM, VSZ and RSS columns.
>
> But being out of memory doesn't necessarily lead to a segfault - it can
> (EG if a malloc failed, and some C programmer neglected to do decent error
> checking), but an OOM kill is more likely.
>

The above can be used to detect a leak in the _process_.

Once it's been established (if it's established) that the process is
getting oversized, you can sometimes see where the memory is going with:
https://www.fugue.co/blog/diagnosing-and-fixing-memory-leaks-in-python.html

But again, a memory leak isn't necessarily going to lead to a segfault.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pytest segfault, not with -v

[Dan Stromberg]

On Sat, Nov 20, 2021 at 10:09 AM Marco Sulla 
wrote:

> I know how to check the refcounts, but I don't know how to check the
> memory usage, since it's not a program, it's a simple library. Is
> there not a way to check inside Python the memory usage? I have to use
> a bash script (I'm on Linux)?
>

ps auxww
...can show you how much memory is in use for the entire process.

It's commonly combined with grep, like:
ps auxww | head -1
ps auxww | grep my-program-name

Have a look at the %MEM, VSZ and RSS columns.

But being out of memory doesn't necessarily lead to a segfault - it can (EG
if a malloc failed, and some C programmer neglected to do decent error
checking), but an OOM kill is more likely.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pytest segfault, not with -v

[Dan Stromberg]

On Fri, Nov 19, 2021 at 9:49 AM Marco Sulla 
wrote:

> I have a battery of tests done with pytest. My tests break with a
> segfault if I run them normally. If I run them using pytest -v, the
> segfault does not happen.
>
> What could cause this quantical phenomenon?
>

Pure python code shouldn't do this, unless you're using ctypes or similar
(which arguably isn't pure python).

But C extension modules sure can.  See:
https://stromberg.dnsalias.org/~strombrg/checking-early.html .  It uses
Fortran to make its point, but the same thing very much applies to C.

BTW, if you're using C extension modules, the troublesome one doesn't
necessarily have to be one you wrote. It could be a dependency created by
someone else too.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: import question

[Dan Stromberg]

On Thu, Nov 18, 2021 at 6:19 PM Chris Angelico  wrote:

> On Fri, Nov 19, 2021 at 11:24 AM Dan Stromberg 
> wrote:
> >
> >
> > On Thu, Nov 18, 2021 at 12:21 PM Chris Angelico 
> wrote:
> >>
> >> If you're trying to make a Python-in-Python sandbox, I recommend not.
> >> Instead, use an OS-level sandbox (a chroot, probably some sort of CPU
> >> usage limiting, etc), and use that to guard the entire Python process.
> >> Python-in-Python will basically *never* be secure.
> >
> >
> > Good advice to not try to sandbox python.
> >
> > But chroot can sometimes be broken out of.  It isn't a cure-all.
> >
>
> That's true, but it's way better than attempting Python-in-Python
> sandboxing. In any case, all the options worth investigating will be
> at the OS level.
>
> (Or maybe higher, but I can't imagine it being practical to create
> individual VMs for each client who comes to the web site.)
>

Actually, there are ports of CPython and Micropython that run inside a web
browser over WASM.  Going with one of these might be safer.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: import question

[Dan Stromberg]

On Thu, Nov 18, 2021 at 12:21 PM Chris Angelico  wrote:

> If you're trying to make a Python-in-Python sandbox, I recommend not.
> Instead, use an OS-level sandbox (a chroot, probably some sort of CPU
> usage limiting, etc), and use that to guard the entire Python process.
> Python-in-Python will basically *never* be secure.
>

Good advice to not try to sandbox python.

But chroot can sometimes be broken out of.  It isn't a cure-all.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to Jupyter Notebook

[Dan Stromberg]

Sorry, gmail got in a hurry to send my message :)

Does this help? https://lwn.net/Articles/855875/

On Sun, Nov 14, 2021 at 4:22 PM Dan Stromberg  wrote:

>
> Does this help?
>
>
> On Wed, Oct 20, 2021 at 8:48 AM Shaozhong SHI 
> wrote:
>
>> Hello,
>>
>> Is anyone familiar with alternatives to Jupyter Notebook.
>>
>> My Jupyter notebook becomes unresponsive in browsers.
>>
>> Are there alternatives to read, edit and run Jupyter Notebook?
>>
>> Regards,
>>
>> David
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Alternatives to Jupyter Notebook

[Dan Stromberg]

Does this help?


On Wed, Oct 20, 2021 at 8:48 AM Shaozhong SHI 
wrote:

> Hello,
>
> Is anyone familiar with alternatives to Jupyter Notebook.
>
> My Jupyter notebook becomes unresponsive in browsers.
>
> Are there alternatives to read, edit and run Jupyter Notebook?
>
> Regards,
>
> David
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

CWD + relative path + import name == resultant relative path?

[Dan Stromberg]

Is there a predefined function somewhere that can accept the 3 things on
the LHS in the subject line, and give back a resultant relative path -
relative to the unchanged CWD?

To illustrate, imagine:
1) You're sitting in /home/foo/coolprog
2) You look up the AST for /home/foo/coolprog/a/b/c.py
3) /home/foo/coolprog/a/b/c.py has a relative import of ..blah

Is there something preexisting that can figure out what path blah is
supposed to have, assuming 100% pure python code?

Or is it necessary to actually import to get that?  I think I could import
and then check __file__, but I'd rather avoid that if possible.

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python script seems to stop running when handling very large dataset

[Dan Stromberg]

On Fri, Oct 29, 2021 at 4:04 PM dn via Python-list 
wrote:

> On 30/10/2021 11.42, Shaozhong SHI wrote:
> > Python script works well, but seems to stop running at a certain point
> when
> > handling very large dataset.
> >
> > Can anyone shed light on this?
>
> Storage space?
> Taking time to load/format/process data-set?
>

It could be many things.

What operating system are you on?

If you're on Linux, you can use strace to attach to a running process to
see what it's up to.  Check out the -p option.  See
https://stromberg.dnsalias.org/~strombrg/debugging-with-syscall-tracers.html

macOS has dtruss, but it's a little hard to enable.  dtruss is similar to
strace.

Both of these tools are better for processes doing system calls (kernel
interactions).  They do not help nearly as much with CPU-bound processes.

It could also be that you're running out of virtual memory, and the
system's virtual memory system is thrashing.

Does the load average on the system go up significantly when the process
seems to get stuck?

You could try attaching to the process with a debugger, too, EG with pudb:
https://github.com/inducer/pudb/issues/31

Barring those, you could sprinkle some print statements in your code, to
see where it's getting stuck. This tends to be an iterative process, where
you add some prints, run, observe the result, and repeat.

HTH.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why so fast a web framework?

[Dan Stromberg]

I care, and I suspect the OP does too.  Usually machine time doesn't matter
as much as developer time, but API overhead Can matter - especially on a
busy server.

I wonder if Pypy would do any better?  Or Micropython?  Or Cython?

CPython != Python.

Sometimes this group reminds me of a certain large company I worked for.
If they didn't have a solution addressing a problem, they'd pretend it
didn't matter and belittle anyone who questioned that version of reality.


On Thu, Oct 28, 2021 at 9:46 AM Calvin Spealman  wrote:

> Who cares?
>
> On Wed, Oct 27, 2021 at 10:47 PM Abdur-Rahmaan Janhangeer <
> arj.pyt...@gmail.com> wrote:
>
> > @Chris @Peter
> >
> >
> > See that famous benchmark
> >
> > https://www.techempower.com/benchmarks/#section=data-r20
> >
> > Like routinely PHP frameworks appear higher up than py
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >
> >
>
> --
>
> CALVIN SPEALMAN
>
> SENIOR QUALITY ENGINEER
>
> calvin.speal...@redhat.com  M: +1.336.210.5107
> [image: https://red.ht/sig] 
> TRIED. TESTED. TRUSTED. 
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: about python

[Dan Stromberg]

Please try to be more specific.

What setup? What problem?

On Thu, Oct 14, 2021 at 10:29 AM Amsalu Fentahun 
wrote:

> there is a problem when I install the setup
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sum() vs. loop

[Dan Stromberg]

On Mon, Oct 11, 2021 at 2:54 PM Steve Keller  wrote:

> I have found the sum() function to be much slower than to loop over the
> operands myself:
>
> def sum_products(seq1, seq2):
> return sum([a * b for a, b in zip(seq1, seq2)])
>
> def sum_products2(seq1, seq2):
> sum = 0
> for a, b in zip(seq1, seq2):
> sum += a * b
> return sum
>
> In a program I generate about 14 million pairs of sequences of ints each
> of length 15 which need to be summed.  The first version with sum() needs
> 44 seconds while the second version runs in 37 seconds.
>
> Can someone explain this difference?
>
I can't explain it.  It might help to try a line-by-line profiler.

If you need speed, maybe try Cython, numpy and/or numba.

It seems like the generator expression should be the fastest to me.  But
writing for speed instead of writing for clarity is usually not a great
idea.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python.exe - System Error

[Dan Stromberg]

Hi.

You'll likely get more help if you include more context, like more text in
your cut and paste.

Also, how did you install it?  And where did you get the installer from?

HTH.

On Thu, Sep 30, 2021 at 8:00 AM jitendrabeura001 
wrote:

>Hello Sir/Madam, please help me to out from this difficulties, whenever
>I'm trying to install python idle it is showing me the error called
>"[1]python.exe - System Error"**
>I have tried couple of ways to get rid out of this but failed every
> time,
>please please help me to find the exact solution for this problem. Thank
>You.
>
> References
>
>Visible links
>1. http://python.exe/
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

McCabe complexity for just changed files in a commit?

[Dan Stromberg]

Hi folks.

Is there a way of getting the McCabe Complexity of just the functions and
methods (in Python) changed in a git commit?

I found radon, and it looks good.  But I think it wants to do entire files,
no?

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Dan Stromberg]

On Thu, Sep 23, 2021 at 8:12 PM Chris Angelico  wrote:

> One good hybrid is to take a subset of Python syntax (so it still
> looks like a Python script for syntax highlighting etc), and then
> parse that yourself, using the ast module. For instance, you can strip
> out comments, then look for "VARNAME = ...", and parse the value using
> ast.literal_eval(), which will give you a fairly flexible file format
> that's still quite safe.
>

Restricting Python with the ast module is interesting, but I don't think
I'd want to bet my career on the actual safety of such a thing.  Given that
Java bytecode was a frequent problem inside web browsers, imagine all the
messiness that could accidentally happen with a subset of Python syntax
from untrusted sources.

ast.literal_eval might be a little better - or a list of such, actually.

Better still to use JSON or ini format - IOW something designed for the
purpose.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: XML Considered Harmful

[Dan Stromberg]

On Tue, Sep 21, 2021 at 7:26 PM Michael F. Stemper <
michael.stem...@gmail.com> wrote:

> If XML is not the way to package data, what is the recommended
> approach?
>

I prefer both JSON and YAML over XML.

XML has both elements and tags, but it didn't really need both. This
results in more complexity than necessary.  Also, XSLT and XPath are not
really all that simple.

But there's hope.  If you're stuck with XML, you can use xmltodict, which
makes XML almost as easy as JSON.

HTH.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Request for argmax(list) and argmin(list)

[Dan Stromberg]

How about this?:

python3 -c 'list_ = [1, 3, 5, 4, 2]; am = max((value, index) for index,
value in enumerate(list_)); print(am)'


On Wed, Sep 1, 2021 at 6:51 AM ABCCDE921 
wrote:

> Because that does 2 passes over the entire array when you only need one
> and there is no option to specify if you want the leftmost or rightmost
> element
>
>
> On Wednesday, September 1, 2021 at 12:02:29 PM UTC+5:30, Paul Bryan wrote:
> > Why not:
> >
> > >>> l = [1, 3, 5, 9, 2, 7]
> > >>> l.index(max(l))
> > 3
> > >>> l.index(min(l))
> > 0
> > On Tue, 2021-08-31 at 21:25 -0700, ABCCDE921 wrote:
> > > I dont want to import numpy
> > >
> > > argmax(list)
> > > returns index of (left most) max element
> > >
> > > argmin(list)
> > > returns index of (left most) min element
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: from foo import bar and the ast module

[Dan Stromberg]

On Sun, Aug 22, 2021 at 7:14 AM Chris Angelico  wrote:

> On Mon, Aug 23, 2021 at 12:08 AM Dan Stromberg 
> wrote:
> >
> > In 'from foo import bar':
> >
> > With the ast module, I see how to get bar, but I do not yet see how to
> get
> > the foo.
> >
> > There are clearly ast.Import and ast.ImportFrom, but I do not see the foo
> > part in ast.ImportFrom.
> >
> > ?
>
> >>> import ast
> >>> ast.dump(ast.parse("from foo import bar"))
> "Module(body=[ImportFrom(module='foo', names=[alias(name='bar')],
> level=0)], type_ignores=[])"
> >>> ast.parse("from foo import bar").body[0].module
> 'foo'
>

With 'from . import bar', I get a module of None.

 Does this seem strange?
-- 
https://mail.python.org/mailman/listinfo/python-list

from foo import bar and the ast module

[Dan Stromberg]

In 'from foo import bar':

With the ast module, I see how to get bar, but I do not yet see how to get
the foo.

There are clearly ast.Import and ast.ImportFrom, but I do not see the foo
part in ast.ImportFrom.

?

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Cyclic imports

[Dan Stromberg]

On Sat, Aug 21, 2021 at 3:35 PM Dan Stromberg  wrote:

> On Tue, Aug 17, 2021 at 11:20 AM Chris Angelico  wrote:
>
>> The main advantage of ast.parse() is that it no longer cares about
>> code layout, and it won't be fooled by an import statement inside a
>> docstring, or anything like that. It's also pretty easy to handle
>> multiple variants (note how "import bar, baz" just has two things in
>> the names list).
>>
>
> I put together something to detect importcycles using the ast module.
> It's at https://stromberg.dnsalias.org/svn/cycles/trunk for now - no
> pypi, no homepage.  I've only run  it on some test inputs, nothing
> substantial. Usage example is in the Makefile.
>
> Imports are kind of complicated. Am I missing any important cases?
>
> Sample output looks like:
> Detected 2 cycles:
> ('mutual_1', 'mutual_2')
> ('directory.mutual_a', 'directory.mutual_b', 'directory.mutual_c')
>
> Thanks!
>

BTW, "readline0.py" is in the project as a Subversion external reference,
so just going to the URL above in a browser won't show it.

But you can install an svn binary and then:
svn checkout https://stromberg.dnsalias.org/svn/cycles/trunk
That should get all of it, including readline0.py
-- 
https://mail.python.org/mailman/listinfo/python-list