[issue44011] Borrow asyncio ssl implementation from uvloop

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29720
pull_request: https://github.com/python/cpython/pull/31597

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46859] NameError: free variable 'outer' referenced before assignment in enclosing scope

[Norman Fung]

Norman Fung  added the comment:

Also, i reverted back to python 3.8.5 and overwrote task.py as recommended.
I think that version we dont already have "GenericAlias".

import asyncio
  File "C:\ProgramData\Anaconda3\lib\asyncio\__init__.py", line 8, in

from .base_events import *
  File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 45, in

from . import staggered
  File "C:\ProgramData\Anaconda3\lib\asyncio\staggered.py", line 11, in

from . import tasks
  File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 20, in 
from types import GenericAlias
ImportError: cannot import name 'GenericAlias' from 'types'
(C:\ProgramData\Anaconda3\lib\types.py)

On Sat, Feb 26, 2022 at 3:20 PM Norman Fung  wrote:

>
> Norman Fung  added the comment:
>
> Thanks Andrew for heads up.
>
> *1. My laptop (Windows 10) *with no changes runs happily with no error from
> here.
> python 3.8.5
> asyncio 3.4.3
>
> 2. *My Windows VM (AWS EC2)* is where the whole mess is happening. I
> *upgraded
> *from Python 3.8.5 to 3.9.7. Asynio stayed 3.4.3, no change.
>
> I manually overwrite C:\ProgramData\Anaconda3\Lib\asyncio\task.py with
> what's here https://github.com/python/cpython/pull/31441/files
> (Only
>
> https://github.com/asvetlov/cpython/blob/150ef068c77abc6a5e7ba97397ac65113dba355a/Lib/asyncio/tasks.py
> )
>
> Before I made this change, the error was:
>   File "src\xxx\xxx.py", line 37, in _invoke_runners
> one_loop.run_until_complete(runner.xxx(xxx, xxx))
>   File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
> 90, in run_until_complete
> self._run_once()
>   File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
> 127, in _run_once
> handle._run()
>   File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
> 196, in run
> ctx.run(self._callback, *self._args)
>   File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 356, in
> _set_state
> _copy_future_state(other, future)
>   File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 335, in
> _copy_future_state
> dest.set_result(result)
>   File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 237, in
> set_result
> self.__schedule_callbacks()
>   File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 149, in
> __schedule_callbacks
> self._loop.call_soon(callback, self, context=ctx)
> Traceback (most recent call last):
>   File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
> 196, in run
> ctx.run(self._callback, *self._args)
>   File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 762, in
> _done_callback
> if outer.done():
> NameError: free variable 'outer' referenced before assignment in enclosing
> scope
>
> After I upgraded python to 3.9.7 and overwrite task.py, a different error
> (but also in asyncio stack):
>
> 2022-02-26 06:48:27,047 casin0: @slack algo_order.id: 13 #8 Algo tick
> level
> error casin0 uat  Non-thread-safe operation invoked
> on an event loop other than the current one Traceback (most recent call
> last):
>   File "C:\dev\xxx.py", line 547, in xxx
> await asyncio.sleep(algo.param.interval_ms / 1000)
>   File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 651, in sleep
> h = loop.call_later(delay,
>   File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 714, in
> call_later
> timer = self.call_at(self.time() + delay, callback, *args,
>   File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 727, in
> call_at
> self._check_thread()
>   File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 785, in
> _check_thread
> raise RuntimeError(
> RuntimeError: Non-thread-safe operation invoked on an event loop other than
> the current one
>
> On Sat, Feb 26, 2022 at 2:36 PM Andrew Svetlov 
> wrote:
>
> >
> > Andrew Svetlov  added the comment:
> >
> > Also, fix is 1 week old. There are no python releases with the fix
> > included yet.
> >
> > On Sat, Feb 26, 2022, 06:58 Norman Fung  wrote:
> >
> > >
> > > Norman Fung  added the comment:
> > >
> > > Thanks Jelle, let me try upgrade first.
> > >
> > > Norman
> > >
> > > On Sat, Feb 26, 2022 at 9:27 AM Jelle Zijlstra  >
> > > wrote:
> > >
> > > >
> > > > Jelle Zijlstra  added the comment:
> > > >
> > > > 3.8 is only receiving security fixes now. Please upgrade. If you
> cannot
> > > > upgrade, I suggest manually applying the patch from
> > > > https://github.com/python/cpython/pull/31441/files to your
> > installation
> > > > of Python.
> > > >
> > > > --
> > > > nosy: +Jelle Zijlstra
> > > > resolution:  -> out of date
> > > > stage:  -> resolved
> > > > status: open -> closed
> > > >
> > > > ___
> > > > Python tracker 
> > > > 
> > > > ___
> > > >
> > >
> > > --
> > >
> > > ___
> > > Python tracker 
> > > 

[issue46430] intern strings in deepfrozen modules

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29719
pull_request: https://github.com/python/cpython/pull/31596

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46859] NameError: free variable 'outer' referenced before assignment in enclosing scope

[Norman Fung]

Norman Fung  added the comment:

Thanks Andrew for heads up.

*1. My laptop (Windows 10) *with no changes runs happily with no error from
here.
python 3.8.5
asyncio 3.4.3

2. *My Windows VM (AWS EC2)* is where the whole mess is happening. I *upgraded
*from Python 3.8.5 to 3.9.7. Asynio stayed 3.4.3, no change.

I manually overwrite C:\ProgramData\Anaconda3\Lib\asyncio\task.py with
what's here https://github.com/python/cpython/pull/31441/files
(Only
https://github.com/asvetlov/cpython/blob/150ef068c77abc6a5e7ba97397ac65113dba355a/Lib/asyncio/tasks.py
)

Before I made this change, the error was:
  File "src\xxx\xxx.py", line 37, in _invoke_runners
one_loop.run_until_complete(runner.xxx(xxx, xxx))
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
90, in run_until_complete
self._run_once()
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
127, in _run_once
handle._run()
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
196, in run
ctx.run(self._callback, *self._args)
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 356, in
_set_state
_copy_future_state(other, future)
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 335, in
_copy_future_state
dest.set_result(result)
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 237, in
set_result
self.__schedule_callbacks()
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 149, in
__schedule_callbacks
self._loop.call_soon(callback, self, context=ctx)
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line
196, in run
ctx.run(self._callback, *self._args)
  File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 762, in
_done_callback
if outer.done():
NameError: free variable 'outer' referenced before assignment in enclosing
scope

After I upgraded python to 3.9.7 and overwrite task.py, a different error
(but also in asyncio stack):

2022-02-26 06:48:27,047 casin0: @slack algo_order.id: 13 #8 Algo tick level
error casin0 uat  Non-thread-safe operation invoked
on an event loop other than the current one Traceback (most recent call
last):
  File "C:\dev\xxx.py", line 547, in xxx
await asyncio.sleep(algo.param.interval_ms / 1000)
  File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 651, in sleep
h = loop.call_later(delay,
  File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 714, in
call_later
timer = self.call_at(self.time() + delay, callback, *args,
  File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 727, in
call_at
self._check_thread()
  File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 785, in
_check_thread
raise RuntimeError(
RuntimeError: Non-thread-safe operation invoked on an event loop other than
the current one

On Sat, Feb 26, 2022 at 2:36 PM Andrew Svetlov 
wrote:

>
> Andrew Svetlov  added the comment:
>
> Also, fix is 1 week old. There are no python releases with the fix
> included yet.
>
> On Sat, Feb 26, 2022, 06:58 Norman Fung  wrote:
>
> >
> > Norman Fung  added the comment:
> >
> > Thanks Jelle, let me try upgrade first.
> >
> > Norman
> >
> > On Sat, Feb 26, 2022 at 9:27 AM Jelle Zijlstra 
> > wrote:
> >
> > >
> > > Jelle Zijlstra  added the comment:
> > >
> > > 3.8 is only receiving security fixes now. Please upgrade. If you cannot
> > > upgrade, I suggest manually applying the patch from
> > > https://github.com/python/cpython/pull/31441/files to your
> installation
> > > of Python.
> > >
> > > --
> > > nosy: +Jelle Zijlstra
> > > resolution:  -> out of date
> > > stage:  -> resolved
> > > status: open -> closed
> > >
> > > ___
> > > Python tracker 
> > > 
> > > ___
> > >
> >
> > --
> >
> > ___
> > Python tracker 
> > 
> > ___
> >
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46712] Share global string identifiers in deepfreeze

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29718
pull_request: https://github.com/python/cpython/pull/31596

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[BELAHCENE Abdelkader]

Thanks every body,
I want to close the subject, but just a naive question:
Does numpy use  a* vectorization *for arrays?
I mean when I add 2 arrays ( or in sum function) how it is done,
in an other word
b=np.arange(100);
 t=np.sum(b)
is equivalent or not to
 s=0
for i in range(100): s +=b[i]
thanks a lot


Le sam. 26 févr. 2022 à 06:44, Dan Stromberg  a écrit :

>
> On Fri, Feb 25, 2022 at 8:12 AM BELAHCENE Abdelkader <
> abdelkader.belahc...@enst.dz> wrote:
>
>> Hi,
>> a lot of people think that C (or C++) is faster than python, yes I agree,
>> but I think that's not the case with numpy, I believe numpy is faster than
>> C, at least in some cases.
>>
>
> This is all "last time I heard".
>
> numpy is written, in significant part, in Fortran.
>
> Fortran, especially for matrix math with variable dimensions, can be
> faster than C.
>
> Fortran, (still last I heard) did not support pointers, which gives
> Fortran compilers the chance to exploit a very nice class of optimizations
> you can't use nearly as well in languages with pointers.
>
> I used to code C to be built with the "noalias" optimization, to get much
> of the speed of Fortran in C.  But it required using an error prone subset
> of C without good error detection.
>
>
>
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46861] os.environ forces variable names to upper case on Windows

[Eryk Sun]

Change by Eryk Sun :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> os.environ should preserve the case of the OS keys ?

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28824] os.environ should preserve the case of the OS keys ?

[Eryk Sun]

Change by Eryk Sun :


--
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46859] NameError: free variable 'outer' referenced before assignment in enclosing scope

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Also, fix is 1 week old. There are no python releases with the fix
included yet.

On Sat, Feb 26, 2022, 06:58 Norman Fung  wrote:

>
> Norman Fung  added the comment:
>
> Thanks Jelle, let me try upgrade first.
>
> Norman
>
> On Sat, Feb 26, 2022 at 9:27 AM Jelle Zijlstra 
> wrote:
>
> >
> > Jelle Zijlstra  added the comment:
> >
> > 3.8 is only receiving security fixes now. Please upgrade. If you cannot
> > upgrade, I suggest manually applying the patch from
> > https://github.com/python/cpython/pull/31441/files to your installation
> > of Python.
> >
> > --
> > nosy: +Jelle Zijlstra
> > resolution:  -> out of date
> > stage:  -> resolved
> > status: open -> closed
> >
> > ___
> > Python tracker 
> > 
> > ___
> >
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45991] Improve ambiguous docstrings in pkgutil

[Barney Gale]

Barney Gale  added the comment:

Should pkgutil call os.fspath() in this case?

--
nosy: +barneygale

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46862] subprocess makes environment blocks with duplicate keys on Windows

[Eryk Sun]

Eryk Sun  added the comment:

I suggest closing this issue as a duplicate of bpo-43702.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43702] [Windows] correctly sort and remove duplicates in _winapi getenvironment()

[Eryk Sun]

Change by Eryk Sun :


--
versions: +Python 3.11 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46862] subprocess makes environment blocks with duplicate keys on Windows

[Eryk Sun]

Eryk Sun  added the comment:

This should be handled in _winapi.CreateProcess(). An environment block is 
technically required to be sorted. (Ages ago this was a MUST requirement for 
getting and setting variables to work correctly, since the implementation 
depended on the sort order, but I think nowadays it's a SHOULD requirement.) 
For example, see the documentation of CreateProcessW() [1]:

If an application provides an environment block, ... explicitly create
these environment variable strings, sort them alphabetically (because
the system uses a sorted environment)

"Changing Environment Variables" is more specific [2]:

All strings in the environment block must be sorted alphabetically by name.
The sort is case-insensitive, Unicode order, without regard to locale.

CompareStringOrdinal() [3] implements a case-insensitive ordinal comparison. 
When a key compares as equal, either keep the current one in the sorted list, 
or replace it with the new key.

---

[1] 
https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessw
[2] 
https://docs.microsoft.com/en-us/windows/win32/procthread/changing-environment-variables
[3] 
https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-comparestringordinal

--
nosy: +eryksun
versions: +Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Dan Stromberg]

On Fri, Feb 25, 2022 at 8:12 AM BELAHCENE Abdelkader <
abdelkader.belahc...@enst.dz> wrote:

> Hi,
> a lot of people think that C (or C++) is faster than python, yes I agree,
> but I think that's not the case with numpy, I believe numpy is faster than
> C, at least in some cases.
>

This is all "last time I heard".

numpy is written, in significant part, in Fortran.

Fortran, especially for matrix math with variable dimensions, can be faster
than C.

Fortran, (still last I heard) did not support pointers, which gives Fortran
compilers the chance to exploit a very nice class of optimizations you
can't use nearly as well in languages with pointers.

I used to code C to be built with the "noalias" optimization, to get much
of the speed of Fortran in C.  But it required using an error prone subset
of C without good error detection.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46854] Failed to compile static python3.7.12

[Ned Deily]

Ned Deily  added the comment:

“I would like to know what environment is the official recommendation for 
static python3.7 compilation.”

There is no official recommendation. As I’ve mentioned, we normally do not 
build with modified Modules/Setup configurations when developing and releasing 
Python. It’s complicated. That’s why setup.py and using Distutils to build the 
standard library modules was introduced years ago. Most users don’t need a 
statically linked Python. Please accept that you need to look elsewhere for 
assistance.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Dan Stromberg]

On Fri, Feb 25, 2022 at 9:03 PM Chris Angelico  wrote:

> On Sat, 26 Feb 2022 at 15:39, Avi Gross via Python-list
>  wrote:
> > Take interpreted languages including Python and R that specify all kinds
> of functions that may be written within the language at first. Someone may
> implement a function like sum() (just an example) that looks like the sum
> of a long list of items is the first item added to a slightly longer sum of
> the remaining items. It stops when the final recursive sum is about to be
> called with no remaining arguments. Clearly this implementation may be a
> tad slow. But does Python require this version of sum() or will it allow
> any version that can be called the same way and returns the same results
> every time?
> >
>
> That's also true of C and pretty much every language I know of. They
> define semantics, not implementation.
>

This comes back to something we've discussed before.

A language that is described primarily by a reference implementation rather
than a standard, runs the risk of being defined by that implementation.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46863] Python 3.10 OpenSSL Configuration Issues

[Adam Pinckard]

New submission from Adam Pinckard :

Python 3.10 does not appear to respecting the OpenSSL configuration within 
linux. Testing completed using Pyenv on both Ubuntu 20.04.4 and Centos-8. Note 
PEP 644 which requires OpenSSL >= 1.1.1 is released in Python 3.10.

We operate behind a corporate proxy / firewall which causes an SSL error where 
the Diffie-Hellman key size is too small. In previous Python versions this is 
resolved by updating the OpenSSL configuration, e.g. downgrading the linux 
crypto policies `sudo update-crypto-policies --set LEGACY`. 

The issue is reproducible in both Ubuntu 20.04.4 and Centos-8. In both linux 
distributions the SSL error is resolvable in earlier Python version, using the 
OpenSSL configurations, but the configuration is not respected with Python 
3.10.2.

See the details below on the kernel versions, linux distributions, and Openssl 
versions, many thanks in advance.

1. Python 3.10.2 Error:
(py_3_10_2) ➜  py_3_10_2 pip install --upgrade pip
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, 
status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: 
DH_KEY_TOO_SMALL] dh key too small (_ssl.c:997)'))': /simple/pip/

2. Ubuntu details
uname -a
Linux Horatio 5.13.0-30-generic #33~20.04.1-Ubuntu SMP Mon Feb 7 14:25:10 UTC 
2022 x86_64 x86_64 x86_64 GNU/Linux

lsb_release  -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 20.04.4 LTS
Release:20.04
Codename:   focal

openssl version -a
OpenSSL 1.1.1f  31 Mar 2020
built on: Wed Nov 24 13:20:48 2021 UTC
platform: debian-amd64
options:  bn(64,64) rc4(16x,int) des(int) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g 
-O2 -fdebug-prefix-map=/build/openssl-dnfdFp/openssl-1.1.1f=. 
-fstack-protector-strong -Wformat -Werror=format-security 
-DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC 
-DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT 
-DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM 
-DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM 
-DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time 
-D_FORTIFY_SOURCE=2
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"
Seeding source: os-specific

2. Centos-8 details
uname -a
Linux localhost.localdomain 5.4.181-1.el8.elrepo.x86_64 #1 SMP Tue Feb 22 
10:00:15 EST 2022 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/centos-release
CentOS Stream release 8

openssl version -a
OpenSSL 1.1.1k  FIPS 25 Mar 2021
built on: Thu Dec  2 16:40:48 2021 UTC
platform: linux-x86_64
options:  bn(64,64) md2(char) rc4(16x,int) des(int) idea(int) blowfish(ptr) 
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -O2 -g -pipe 
-Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS 
-fexceptions -fstack-protector-strong -grecord-gcc-switches 
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic 
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection 
-Wa,--noexecstack -Wa,--generate-missing-build-notes=yes 
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -DOPENSSL_USE_NODELETE -DL_ENDIAN 
-DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT 
-DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM 
-DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM 
-DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG 
-DPURIFY -DDEVRANDOM="\"/dev/urandom\"" 
-DSYSTEM_CIPHERS_FILE="/etc/crypto-policies/back-ends/openssl.config"
OPENSSLDIR: "/etc/pki/tls"
ENGINESDIR: "/usr/lib64/engines-1.1"
Seeding source: os-specific
engines:  rdrand dynamic

--
assignee: christian.heimes
components: SSL
messages: 414072
nosy: adam, christian.heimes
priority: normal
severity: normal
status: open
title: Python 3.10 OpenSSL Configuration Issues
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46858] mmap constructor resets the file pointer on Windows

[Eryk Sun]

Eryk Sun  added the comment:

The resize() method also modifies the file pointer. Instead of fixing that 
oversight, I think it should directly set the file's FileEndOfFileInfo and 
FileAllocationInfo. For example:

// resize the file
if (!SetFileInformationByHandle(
self->file_handle, FileEndOfFileInfo,
_size, sizeof(max_size)) ||
  !SetFileInformationByHandle(
self->file_handle, FileAllocationInfo,
_size, sizeof(max_size)))
{
// resizing failed. try to remap the file
file_resize_error = GetLastError();
max_size.QuadPart = self->size;
new_size = self->size;
}

This is cheaper in terms of system calls. The existing implementation makes 
four system calls: one to set the file pointer in SetFilePointerEx() and three 
in SetEndOfFile(), which queries the file pointer, sets the end-of-file info, 
and sets the allocation info. 

Note that this approach doesn't modify the file pointer in any case. This may 
be surprising if the file size shrinks to less than the existing file pointer. 
But os.ftruncate() behaves the same way, as does the resize() method in Linux.

--
nosy: +eryksun
versions:  -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Chris Angelico]

On Sat, 26 Feb 2022 at 15:39, Avi Gross via Python-list
 wrote:
> Take interpreted languages including Python and R that specify all kinds of 
> functions that may be written within the language at first. Someone may 
> implement a function like sum() (just an example) that looks like the sum of 
> a long list of items is the first item added to a slightly longer sum of the 
> remaining items. It stops when the final recursive sum is about to be called 
> with no remaining arguments. Clearly this implementation may be a tad slow. 
> But does Python require this version of sum() or will it allow any version 
> that can be called the same way and returns the same results every time?
>

That's also true of C and pretty much every language I know of. They
define semantics, not implementation.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46859] NameError: free variable 'outer' referenced before assignment in enclosing scope

[Norman Fung]

Norman Fung  added the comment:

Thanks Jelle, let me try upgrade first.

Norman

On Sat, Feb 26, 2022 at 9:27 AM Jelle Zijlstra 
wrote:

>
> Jelle Zijlstra  added the comment:
>
> 3.8 is only receiving security fixes now. Please upgrade. If you cannot
> upgrade, I suggest manually applying the patch from
> https://github.com/python/cpython/pull/31441/files to your installation
> of Python.
>
> --
> nosy: +Jelle Zijlstra
> resolution:  -> out of date
> stage:  -> resolved
> status: open -> closed
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46854] Failed to compile static python3.7.12

[aprpp]

aprpp <916495...@qq.com> added the comment:

I'm really sorry, I would like to know what environment is the official 
recommendation for static python3.7 compilation? What are the steps and 
commands for compilation officially required? Because the static python3.7 is 
very important to me, it has been a week and it has not been compiled 
successfully.Hope to get official help.Thanks

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Avi Gross via Python-list]

Agreed, Chris. There are many ways to get something done. I often use the 
Anaconda distribution because it tends to bundle many of the modules I need and 
more.

Not that it is such a big deal to load the ones you need, but if you share your 
program, others trying to use it may have some problems.


-Original Message-
From: Chris Angelico 
To: python-list@python.org 
Sent: Fri, Feb 25, 2022 11:16 pm
Subject: Re: C is it always faster than nump?


On Sat, 26 Feb 2022 at 14:35, Avi Gross via Python-list
 wrote:
> But with numpy and more available anyway, it may not be necessary to reinvent 
> much of that. I was just wondering if it ever made sense to simply include it 
> in the base python, perhaps as a second executable with a name like pythonn 
> to signify that it is more numeric. So if you run that, you know you do not 
> need to add an assortment of modules. I keep seeing programs that just 
> automatically add numpy and pandas and various graphic modules and other 
> scientific and machine learning modules. Of course not everyone needs or even 
> wants this. Many simply use base Python techniques even if they are low for 
> larger amounts of data.
>

How would that be different from getting one of the numeric/scientific
distributions of Python? Why should it be a different Python
executable?!?

ChrisA

-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Avi Gross via Python-list]

Yes, Chris, C is real as a somewhat abstract concept. There are a whole slew of 
different variations each time it is released anew with changes and then some 
people at various times built actual compilers that implement a varying subset 
of what is possible, and not necessarily in quite the same way.

As you gathered, I am saying that comparing languages is not so effective as 
comparing implementations and even better specific programs on specific data. 
And yet, you can still get odd results if you cherry pick what to test. 
Consider a sorting algorithm that rapidly checks if the data is already sorted, 
and if so, does not bother sorting it. It will quite possibly be the fastest 
one in a comparison if the data is chosen to be already in order! But on many 
other sets of data it will have wasted some time checking if it is in order 
while other algorithms have started sorting it!

Bad example, maybe, but there are better ones. Consider an algorithm that does 
no checking for one of many errors that can happen. It does not see if the 
arguments it gets are within expected ranges of types or values. It does not 
intercept attempts to divide by zero and much more. Another algorithm is quite 
bulletproof and thus has lots more code and maybe runs much slower. Is it 
shocking if it tests slower . But the other code may end up failing faster in 
the field and need a rewrite.

A really fair comparison is often really hard. Languages are abstract and 
sometimes a new implementation makes a huge change.

Take interpreted languages including Python and R that specify all kinds of 
functions that may be written within the language at first. Someone may 
implement a function like sum() (just an example) that looks like the sum of a 
long list of items is the first item added to a slightly longer sum of the 
remaining items. It stops when the final recursive sum is about to be called 
with no remaining arguments. Clearly this implementation may be a tad slow. But 
does Python require this version of sum() or will it allow any version that can 
be called the same way and returns the same results every time? Does it even 
matter if the function is written in C or C++ or FORTRAN or even assembler of 
some kind, as long as it is placed in an accessible library and there is some 
interface that allows you to make the call in python notation and it is fed to 
the function in the way it requires, and similarly deals with returned values? 
A wrapper, sort of.

The use of such a shortcut is not against the spirit of the language. You can 
still specify you want the sum() function from some module, or write your own. 
This is true most places. I remember way back when how early UNIX shells did 
silly things like call /bin/echo to do trivial things, or call an external 
program to do something as trivial as i=i+1 and then they started building in 
such functionality and your shell scripts suddenly really speeded up. A 
non-programmer I once worked for wrote some truly humongous shell scripts that 
brought machines it was run on remotely in places like Japan during their 
day-time to their knees. Collecting billing data from all over by running a 
pipeline with 9 processes per line/row was a bit much. 

At first I sped it up quite a bit by using newer built-in features like I 
described, or doing more with fewer elements in pipelines. But I saw how much 
was caused by using the wrong tools for the job and there were programs 
designed to analyze data in various ways.

I replaced almost all of it with an AWK script that speeded things up many 
orders of magnitude. And, yes, AWK was not as fast as C but more trivial to 
program in for this need as it had so  many needed aspects built-in or 
happening automagically.

Would we do the entire project differently today? Definitely. All the billing 
records would not be sitting in an assortment of flat files all over the place 
but rather be fed into some database that made retrieval of all kinds of 
reports straightforward without needing to write much code at all.

How many modules or "packages" were once written largely using the language and 
then gradually "improved" by replacing parts, especially slower parts, with 
external content as we have been discussing? In a sense, some Python 
applications run on older versions of Python may be running faster as newer 
versions have improved some of the "same" code while to the user, they see it 
running on the same language, Python?

-Original Message-
From: Chris Angelico 
To: python-list@python.org 
Sent: Fri, Feb 25, 2022 2:58 pm
Subject: Re: C is it always faster than nump?


On Sat, 26 Feb 2022 at 06:44, Avi Gross via Python-list
 wrote:
>
> I agree with Richard.
>
> Some people may be confused and think c is the speed of light and 
> relativistically speaking, nothing can be faster. (OK, just joking. The uses 
> of the same letter of the alphabet are not at all related. One is named for 
> the language that came after 

[issue46862] subprocess makes environment blocks with duplicate keys on Windows

[benrg]

New submission from benrg :

On Windows, if one writes

env = os.environ.copy()
env['http_proxy'] = 'whatever'

or either of the documented equivalents ({**os.environ, ...} or (os.environ | 
{...})), and passes the resulting environment to subprocess.run or 
subprocess.Popen, the spawned process may get an environment containing both 
`HTTP_PROXY` and `http_proxy`. Most Win32 software will see only the first one, 
which contains the unmodified value from os.environ.

Because os.environ forces all keys to upper case, it's possible to work around 
this by using only upper case keys in the update, but that behavior of 
os.environ is nonstandard (issue 46861), and subprocess shouldn't depend on it 
always being true, nor should end users have to.

Since dicts preserve order, the user's (presumable) intent is preserved in the 
env argument. I think subprocess should do something like

env = {k.upper(): (k, v) for k, v in env.items()}
env = dict(env.values())

to discard duplicate keys, keeping only the rightmost one.

--
components: Library (Lib), Windows
messages: 414068
nosy: benrg, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: subprocess makes environment blocks with duplicate keys on Windows
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Chris Angelico]

On Sat, 26 Feb 2022 at 14:35, Avi Gross via Python-list
 wrote:
> But with numpy and more available anyway, it may not be necessary to reinvent 
> much of that. I was just wondering if it ever made sense to simply include it 
> in the base python, perhaps as a second executable with a name like pythonn 
> to signify that it is more numeric. So if you run that, you know you do not 
> need to add an assortment of modules. I keep seeing programs that just 
> automatically add numpy and pandas and various graphic modules and other 
> scientific and machine learning modules. Of course not everyone needs or even 
> wants this. Many simply use base Python techniques even if they are low for 
> larger amounts of data.
>

How would that be different from getting one of the numeric/scientific
distributions of Python? Why should it be a different Python
executable?!?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45735] Promise the long-time truth that `args=list` works

[Tim Peters]

Tim Peters  added the comment:


New changeset e466faa9df9a1bd377d9725de5484471bc4af8d0 by Charlie Zhao in 
branch 'main':
bpo-45735: Promise the long-time truth that `args=list` works (GH-30982)
https://github.com/python/cpython/commit/e466faa9df9a1bd377d9725de5484471bc4af8d0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46854] Failed to compile static python3.7.12

[Ned Deily]

Ned Deily  added the comment:

P.S. One other suggestion: try building a non-static version of Python 3.7 
first, i.e. without modifying Modules/Setup, and, once you get that working, it 
should be easier to examine the output from the build and copy those paths that 
setup.py found into your Modules/Setup.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46854] Failed to compile static python3.7.12

[Ned Deily]

Ned Deily  added the comment:

Sorry, I'll repeat: "Successfully building a statically-linked version of 
Python depends greatly on the environment in which you are trying to build it 
and is really beyond the scope of this bug tracker." Note, a quick web search 
found this page which should give you some suggestions on what dev packages 
need to be installed in an Ubuntu 18.04 environment. But also note that the 
directions there are for a normal, non-static Python build using setup.py and 
not Modules/Setup which, again, we don't normally test for 3.7. Good luck!

https://linuxize.com/post/how-to-install-python-3-7-on-ubuntu-18-04/

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46861] os.environ forces variable names to upper case on Windows

[benrg]

New submission from benrg :

The Windows functions that deal with environment variables are case-insensitive 
and case-preserving, like most Windows file systems. Many environment variables 
are conventionally written in all caps, but others aren't, such as 
`ProgramData`, `PSModulePath`, and `windows_tracing_logfile`.

os.environ forces all environment variable names to upper case when it's 
constructed. One consequence is that if you pass a modified environment to 
subprocess.Popen, you end up with variables named `PROGRAMDATA`, etc., even if 
you didn't modify their values.

While this is unlikely to break things since other software normally ignores 
the case, it's nonstandard behavior, and disconcerting when the affected 
variable names are shown to human beings.

Here's an example of someone being confused by this: 
https://stackoverflow.com/questions/19023238/why-python-uppercases-all-environment-variables-in-windows

--
components: Library (Lib), Windows
messages: 414064
nosy: benrg, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: os.environ forces variable names to upper case on Windows
type: behavior
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[Abdur-Rahmaan Janhangeer]

> I never knew this. Where can I read more about this origin?

https://python-history.blogspot.com/2009/01/personal-history-part-1-cwi.html?m=1

<>


There are more sources which i cannot remember
from the top of my head
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46854] Failed to compile static python3.7.12

[aprpp]

aprpp <916495...@qq.com> added the comment:

As shown in the attachment, how should I add the definition of module _ctypes 
to Modules/Setup? Modules like _ctypes are not given in Modules/Setup.dist, I 
really don't know what they depend on

--
Added file: https://bugs.python.org/file50644/Setup.PNG

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Avi Gross via Python-list]

Dennis,

What you describe may be a start but is it anything I might not have easily 
created myself? https://docs.python.org/3/library/array.html

I can see creating my own object and adding those methods and attributes while 
gaining very little, except perhaps storage. 

Can I add or multiply two such items efficiently if it contains a numeric 
value? Can I offer them as an argument to all kinds of functions which can now 
handle it well? How does it work if a second operand is a scalar or an array of 
another data type. Can two be compared and result in an array of boolean (not 
seen in the list of types). Numpy does quite a bit of that kind of thing but 
perhaps better is a language like R where all that and more are built in. 

But with numpy and more available anyway, it may not be necessary to reinvent 
much of that. I was just wondering if it ever made sense to simply include it 
in the base python, perhaps as a second executable with a name like pythonn to 
signify that it is more numeric. So if you run that, you know you do not need 
to add an assortment of modules. I keep seeing programs that just automatically 
add numpy and pandas and various graphic modules and other scientific and 
machine learning modules. Of course not everyone needs or even wants this. Many 
simply use base Python techniques even if they are low for larger amounts of 
data.



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Oscar Benjamin]

On Sat, 26 Feb 2022 at 03:10, Dennis Lee Bieber  wrote:
>
> On Fri, 25 Feb 2022 23:06:57 + (UTC), Avi Gross 
> declaimed the following:
>
> >I do have to wonder if anyone ever considered adding back enough 
> >functionality into base Python to make some additions less needed. Is there 
> >any reason the kind of structures used by so many languages cannot be made 
> >part of python such as a vector/array that holds exactly one kind of data 
> >structure and not force use of things like a list when that is more than is 
> >needed?
> >
> https://docs.python.org/3/library/array.html
>
> seems to fit the criteria...

The stdlib array module is basically unused in comparison to NumPy.
The capabilities of the array module do not meet the needs for most
users who want to do anything useful with arrays.

The intention in creating NumPy  (in the NumPy/SciPy split) was that
it might be possible that NumPy could be merged into core Python.
Unfortunately that didn't come to be.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46857] Python leaks one reference at exit on Windows

[Jeremy Kloth]

Jeremy Kloth  added the comment:

> ./configure --enabled-shared --with-py-debug --with-trace-refs

(that's what I get for typing from memory):
./configure --enable-shared --with-pydebug --with-trace-refs

> > I proposed GH-31594 to fix this macro.
>
> Even using that change, I still have negative refs (but I still have
> Py_TRACE_REFS defined)

I initially missed the _PySet_Dummy change, with that total refs (w/o
dump_refs) is now 0.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46854] Failed to compile static python3.7.12

[aprpp]

aprpp <916495...@qq.com> added the comment:

The platform i build python3.7.12 on is Ubuntu18.04, I would like to know how 
the modules I fail to build, like _ctypes, should be added in Modules/Setup.
The steps for my build are：
1.tar -zxvf Python-3.7.12.tgz && cd Python-3.7.12 
2.edit Modules/Setup, add modules build definitions in the Modules/Setup
3../configure LDFLAGS="-static" --disable-shared --prefix=/home/python3
4.make LDFLAGS="-static" LINKFORSHARED=" "

How to build python3.7 statically without relying on editing Modules/Setup？

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[Abdur-Rahmaan Janhangeer]

Yes i know that JS has switch from before

I was referring to the pattern matching proposal.

I cannot find the original place i read it from but here's the github one

https://github.com/tc39/proposal-pattern-matching
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46857] Python leaks one reference at exit on Windows

[Jeremy Kloth]

Jeremy Kloth  added the comment:

> Oh wow. How did you find this leak? Did you read all C files and check for 
> code specific to Windows? How did you proceed? Well spotted!

Initially, I modified Py_INCREF to dump the object (addr & tp_name) on
initial inc (ob_refcnt == 1) and Py_DECREF to dump on final dec
(ob_refcnt == 0).
Then filter that list (~65K) to find objects not dealloc'ed.  Given
those names (~200), cross-check with source files containing 'ifdef
MS_WINDOWS' (and related spellings).

> Which command do you type? Do you pass -I option to Python?

For both as -I disables environment lookup:
--- a/Python/initconfig.c
+++ b/Python/initconfig.c
@@ -757,6 +757,7 @@ config_init_defaults(PyConfig *config)
config->user_site_directory = 1;
config->buffered_stdio = 1;
config->pathconfig_warnings = 1;
+   config->dump_refs = 1;
#ifdef MS_WINDOWS
config->legacy_windows_stdio = 0;
#endif

For linux:
./configure --enabled-shared --with-py-debug --with-trace-refs
make build_all
LD_LIBRARY_PATH=$PWD ./python -X showrefcount -I -c pass

For Windows:
Add "#define Py_TRACE_REFS 1" to PC\pyconfig.h
build.bat -d -e
amd64\python_d.exe -X showrefcount -I -c pass

> I proposed GH-31594 to fix this macro.

Even using that change, I still have negative refs (but I still have
Py_TRACE_REFS defined)

--
nosy: +jeremy.kloth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46854] Failed to compile static python3.7.12

[aprpp]

Change by aprpp <916495...@qq.com>:


--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Dennis Lee Bieber]

On Fri, 25 Feb 2022 23:06:57 + (UTC), Avi Gross 
declaimed the following:

>I do have to wonder if anyone ever considered adding back enough functionality 
>into base Python to make some additions less needed. Is there any reason the 
>kind of structures used by so many languages cannot be made part of python 
>such as a vector/array that holds exactly one kind of data structure and not 
>force use of things like a list when that is more than is needed?
>
https://docs.python.org/3/library/array.html

seems to fit the criteria...


-- 
Wulfraed Dennis Lee Bieber AF6VN
wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Oscar Benjamin]

On Fri, 25 Feb 2022 at 23:13, Barry  wrote:
>
> > On 25 Feb 2022, at 23:00, Richard Damon  wrote:
> >
> > On 2/25/22 2:47 PM, Chris Angelico wrote:
> >>> On Sat, 26 Feb 2022 at 05:49, Richard Damon  
> >>> wrote:
> >>> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
>  Hi,
>  a lot of people think that C (or C++) is faster than python, yes I agree,
>  but I think that's not the case with numpy, I believe numpy is faster 
>  than
>  C, at least in some cases.
> 
> >>> My understanding is that numpy is written in C, so for it to be faster
> >>> than C, you are saying that C is faster that C.
> >> Fortran actually, but ultimately, they both get compiled to machine code.
> >
> > Looking at the Github repo I see:
> >
> > Languages:
> > Python.  62.5%
> > C   35.3%
> > C++.   1.0%
> > Cython.   0.9%
> > Shell.   0.2%
> > Fortran.   0.1%
>
> I assume that this is just for bumpy and not for all its dependencies.
> That will add a lot of Fortran and c++ I expect.

NumPy links with BLAS/LAPACK that will do the heavy lifting for common
linear algebra operations. Multiple different BLAS libraries can be
used with NumPy and those libraries might be written in Fortran and
might also involve some hand-crafted assembly for particular
architectures. Some explanation of NumPy's BLAS/LAPACK support is
here:
https://numpy.org/devdocs/user/building.html

By default if you install NumPy from conda (or at least if you install
the Anaconda distribution) then I think that NumPy will use the Intel
MKL library:
https://en.wikipedia.org/wiki/Math_Kernel_Library
As I understand it the core of MKL is Fortran and is compiled with
Intel's ifortran compiler but some parts might be C/C++. MKL is also
the same library that is used by Matlab for its own heavy lifting.
It's not sufficiently free to be used in NumPy's PyPI wheels though.

If you install the precompiled NumPy wheels with pip from PyPI then I
think those are statically linked with OpenBLAS:
https://github.com/xianyi/OpenBLAS
Again I think the core of OpenBLAS is Fortran but there's some C in there.

In the pre-wheel days the situation was that NumPy provided installer
files for Windows that would give binaries linked with ATLAS (also
Fortran):
https://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Algebra_Software

I think at some point NumPy used to use the OSX Accelerate library but
the page I linked above says that's now deprecated. I don't know
anything about Accelerate but I wouldn't be surprised to hear that it
was a bunch of old Fortran code!

If you build NumPy from source without having any BLAS/LAPACK
libraries then I think it uses its own backup version of these that is
written in C but not as well optimised. This used to be the default
for a pip install on Linux in pre-wheel times.

Many operations in NumPy don't actually use BLAS/LAPACK and for those
parts the heavy lifting is all done in NumPy's own C code.

Lastly SciPy which is very often used together with NumPy does have a
load of Fortran code. As I understand it at some point NumPy and SciPy
were divided from the previous numerical Python libraries and there
was a deliberate split so that the Fortran code all ended up in SciPy
rather than NumPy.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46854] Failed to compile static python3.7.12

[Ned Deily]

Ned Deily  added the comment:

Successfully building a statically-linked version of Python depends greatly on 
the environment in which you are trying to build it and is really beyond the 
scope of this bug tracker. You don't say on which OS platform and version you 
are attempting this so it is hard to give any meaningful suggestions other than 
to make sure you have installed development versions (those including header 
files) of the various third-party libraries Python builds depend on (like 
libssl and libcrypto from OpenSSL), if your platform has a package manager, or 
build the required third-party libraries yourself. In any case, you will 
probably need to further edit the Setup file to reflect the location of the 
various third-party headers and libraries. As a rule, we don't do a lot of 
testing of static builds using Setup directly but some of the downstream 
distributors of Python do. Also note that Python 3.7 is now in the 
security-fix-only phase of its lifecycle prior to end-of-life in 2023, so 
further changes 
 to the build process in 3.7 at this point would be out-of-scope. For what it's 
worth, there has been a significant amount of work in this area that is going 
into the next Python feature release, 3.11.0, planned for later this year so it 
should be significantly easier to do static builds and the like in 3.11. If you 
still run into problems with 3.7, suggest you ask in help forums like 
StackOverflow.  Good luck!

--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46857] Python leaks one reference at exit on Windows

[STINNER Victor]

STINNER Victor  added the comment:

> Good news, the difference on Windows was easy enough to find, bad news total 
> refs are now negative!

Oh wow. How did you find this leak? Did you read all C files and check for code 
specific to Windows? How did you proceed? Well spotted!


>  #define INIT_ALIAS(NAME, TYPE)

I proposed GH-31594 to fix this macro.


> Strange as well, when using dump_refs, the total refs are much more negative 
> (-12 linux, -13 Windows)

Which command do you type? Do you pass -I option to Python?

With my PR, I get exactly 0 on Linux:

$ ./python -I -X showrefcount -c pass
[0 refs, 0 blocks]


> Note that an allocated block is still leaking.

Right, with my PR, I now get 1 leaked memory block on Windows:

> python -I -X showrefcount -c pass   
[0 refs, 1 blocks]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46857] Python leaks one reference at exit on Windows

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29717
pull_request: https://github.com/python/cpython/pull/31594

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46857] Python leaks one reference at exit on Windows

[Jeremy Kloth]

Jeremy Kloth  added the comment:

Note that an allocated block is still leaking.

Strange as well, when using dump_refs, the total refs are much more negative 
(-12 linux, -13 Windows)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46857] Python leaks one reference at exit on Windows

[Jeremy Kloth]

Jeremy Kloth  added the comment:

Good news, the difference on Windows was easy enough to find, bad news total 
refs are now negative!

--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -3647,8 +3647,7 @@ _PyBuiltins_AddExceptions(PyObject *bltinmod)

 #define INIT_ALIAS(NAME, TYPE) \
 do { \
-Py_INCREF(PyExc_ ## TYPE); \
-Py_XDECREF(PyExc_ ## NAME); \
+Py_XSETREF(PyExc_ ## NAME, PyExc_ ## TYPE); \
 PyExc_ ## NAME = PyExc_ ## TYPE; \
 if (PyDict_SetItemString(mod_dict, # NAME, PyExc_ ## NAME)) { \
 return -1; \

As the PyExc_* aliases just deprecated names for PyExc_OSError, there is no 
need to increment their refcounts.  Or they could be decremented in Fini().  Or 
they could finally be removed entirely.

--
nosy: +jkloth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46859] NameError: free variable 'outer' referenced before assignment in enclosing scope

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

3.8 is only receiving security fixes now. Please upgrade. If you cannot 
upgrade, I suggest manually applying the patch from 
https://github.com/python/cpython/pull/31441/files to your installation of 
Python.

--
nosy: +Jelle Zijlstra
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46854] Failed to compile static python3.7.12

[aprpp]

aprpp <916495...@qq.com> added the comment:

I  have signed the contributor agreement

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46748] Python.h includes stdbool.h

[STINNER Victor]

STINNER Victor  added the comment:

> New changeset 2c228a7b8f89e9ed8d390370abd771d4993b79d8 by Petr Viktorin in 
> branch 'main':
> bpo-46748: Don't import  in public headers (GH-31553)

It seems like this change broke ctypes on some architectures like s390x:

https://buildbot.python.org/all/#/builders/3/builds/1573

==
FAIL: test_frozentable (ctypes.test.test_values.PythonValuesTestCase) 
[__phello_alias__]
--
Traceback (most recent call last):
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/ctypes/test/test_values.py",
 line 86, in test_frozentable
self.assertIsNone(spec.submodule_search_locations)
^^
AssertionError: [] is not None
==
FAIL: test_frozentable (ctypes.test.test_values.PythonValuesTestCase) 
[__phello__]
--
Traceback (most recent call last):
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/ctypes/test/test_values.py",
 line 86, in test_frozentable
self.assertIsNone(spec.submodule_search_locations)
^^
AssertionError: 
['/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/__phello__']
 is not None
==
FAIL: test_frozentable (ctypes.test.test_values.PythonValuesTestCase) 
[__phello__.ham]
--
Traceback (most recent call last):
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/ctypes/test/test_values.py",
 line 86, in test_frozentable
self.assertIsNone(spec.submodule_search_locations)
^^
AssertionError: 
['/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang/build/Lib/__phello__/ham']
 is not None

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[Greg Ewing]

On 26/02/22 11:22 am, Chris Angelico wrote:

What's the best language for swearing in? Ah, that one I can't help
you with, although I've heard good things about French. :)


Russian seems to be quite good for it too, judging by
certain dashcam footage channels.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

[issue46838] Parameters and arguments parser syntax error improvments

[Andrej Klychin]

Andrej Klychin  added the comment:

@terry.reedy, I based that error message on current >>> foo(**{}, *())
SyntaxError: iterable argument unpacking follows keyword argument unpacking

and >>> foo(__debug__=True)
SyntaxError: cannot assign to __debug__

but the final error message could be anything if it explicitly says "what's 
wrong".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46838] Parameters and arguments parser syntax error improvments

[Andre Roberge]

Change by Andre Roberge :


--
nosy: +aroberge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46860] `--with-suffix` not respected on case-insensitive file systems

[Brett Cannon]

Change by Brett Cannon :


--
keywords: +patch
pull_requests: +29716
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31593

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46860] `--with-suffix` not respected on case-insensitive file systems

[Brett Cannon]

New submission from Brett Cannon :

If you use `--with-suffix` on a case-insensitive file system it is ultimately 
ignored and forced to `.exe`.

PR incoming.

--
assignee: brett.cannon
components: Build
messages: 414051
nosy: brett.cannon
priority: normal
severity: normal
status: open
title: `--with-suffix` not respected on case-insensitive file systems
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove the float.__setformat__() method

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29715
pull_request: https://github.com/python/cpython/pull/31592

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46852] Remove the float.__setformat__() method

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5ab745fc51e159ead28b523414e52f0bcc1ef353 by Victor Stinner in 
branch 'main':
bpo-46852: Remove the float.__set_format__() method (GH-31585)
https://github.com/python/cpython/commit/5ab745fc51e159ead28b523414e52f0bcc1ef353


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[STINNER Victor]

STINNER Victor  added the comment:

I close again the issue, the C API should now be fine :-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Brandt Bucher]

Change by Brandt Bucher :


--
pull_requests: +29714
pull_request: https://github.com/python/cpython/pull/31591

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46859] NameError: free variable 'outer' referenced before assignment in enclosing scope

[Norman Fung]

New submission from Norman Fung :

In reference to ticket (which was fix for Python 3.9 or above) 
https://bugs.python.org/issue46672?@ok_message=msg%20413975%20created%0Aissue%2046672%20nosy_count%2C%20nosy%2C%20messages%2C%20message_count%20edited%20ok&@template=item

I encountered this problem on:
a) Python 3.8.5
b) asyncio 3.4.3

Stack
Exception in callback gather.._done_callback() at C:\ProgramData\Anaconda3\lib\asyncio\tasks.py:758
handle: ._done_callback() 
at C:\ProgramData\Anaconda3\lib\asyncio\tasks.py:758 created at 
C:\ProgramData\Anaconda3\lib\asyncio\futures.py:149>
source_traceback: Object created at (most recent call last):
  File "src\x.py", line 37, in _invoke_runners
one_loop.run_until_complete(runner.x)
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line 90, 
in run_until_complete
self._run_once()
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line 127, 
in _run_once
handle._run()
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line 196, 
in run
ctx.run(self._callback, *self._args)
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 356, in 
_set_state
_copy_future_state(other, future)
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 335, in 
_copy_future_state
dest.set_result(result)
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 237, in 
set_result
self.__schedule_callbacks()
  File "C:\ProgramData\Anaconda3\lib\asyncio\futures.py", line 149, in 
__schedule_callbacks
self._loop.call_soon(callback, self, context=ctx)
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\nest_asyncio.py", line 196, 
in run
ctx.run(self._callback, *self._args)
  File "C:\ProgramData\Anaconda3\lib\asyncio\tasks.py", line 762, in 
_done_callback
if outer.done():
NameError: free variable 'outer' referenced before assignment in enclosing scope

--
components: asyncio
messages: 414048
nosy: asvetlov, miss-islington, norman.lm.fung, onerandomusername, sobolevn, 
yselivanov
priority: normal
severity: normal
status: open
title: NameError: free variable 'outer' referenced before assignment in 
enclosing scope
type: crash
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46816] Remove declarations for non-__STDC__ compilers

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4060111f9dc44682f9d7bdafb4e7dacb96706ad3 by Oleg Iarygin in 
branch 'main':
bpo-46816: Remove declarations for non-__STDC__ compilers (GH-31466)
https://github.com/python/cpython/commit/4060111f9dc44682f9d7bdafb4e7dacb96706ad3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46816] Remove declarations for non-__STDC__ compilers

[STINNER Victor]

STINNER Victor  added the comment:

It should be fine :-D Thanks.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: One-liner to merge lists?

[Chris Angelico]

On Sat, 26 Feb 2022 at 10:05, Albert-Jan Roskam  wrote:
>Was also thinking about reduce, though this one uses a dunder method:
>from functools import reduce
>d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>print(reduce(list.__add__, list(d.values(

If you don't want to use the dunder, just use the operator module:

reduce(operator.add, d.values())

But ultimately, that's still the same as sum(), and it's no more
readable than chain(), so I'd still be inclined to go with chain and
then wrap it in a function if you need the clarity.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Barry]

> On 25 Feb 2022, at 23:00, Richard Damon  wrote:
> 
> On 2/25/22 2:47 PM, Chris Angelico wrote:
>>> On Sat, 26 Feb 2022 at 05:49, Richard Damon  
>>> wrote:
>>> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
 Hi,
 a lot of people think that C (or C++) is faster than python, yes I agree,
 but I think that's not the case with numpy, I believe numpy is faster than
 C, at least in some cases.
 
>>> My understanding is that numpy is written in C, so for it to be faster
>>> than C, you are saying that C is faster that C.
>> Fortran actually, but ultimately, they both get compiled to machine code.
> 
> Looking at the Github repo I see:
> 
> Languages:
> Python.  62.5%
> C   35.3%
> C++.   1.0%
> Cython.   0.9%
> Shell.   0.2%
> Fortran.   0.1%

I assume that this is just for bumpy and not for all its dependencies.
That will add a lot of Fortran and c++ I expect.

> 
> So there is a bit of Fortan in there, but it looks like most of the heavy 
> lifting is in C.
> 
> My guess is the Fortran is likely some hooks to add Fortran modules into the 
> program with numpy.
> 
> ...
>>> The key point is that numpy was written by skilled programmers who
>>> carefully optimized their code to be as fast as possible for the major
>>> cases. Thus it is quite possible for the numpy code to be faster in C
>>> than code written by a person without that level of care and effort.
>> This is clearly true.
>> 
>> ChrisA
> 
> 
> -- 
> Richard Damon
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Avi Gross via Python-list]

Is that fair, Grant?

I go back far enough that in my earliest years I was submitting FORTRAN 
programs written on punched cards and often getting them back the next day. The 
compiling was not the major factor in how long it took.

For many cases, a compiled language only needs to be compiled once and can be 
run many times. Other than during development timeframes, the major concept of 
speed is how well it runs not how long it takes to compile, especially if the 
compiler is busy doing all kinds of optimizations to your code or checking for 
possible obscure errors or making sure the code does not try something funny 
like access memory not allowed and so on.

An interpreted language often has to do the same things every time, albeit some 
have come up with ways to partially compile modules and only redo if they have 
been changed. Some of what they do every time is in some sense wasted effort 
but as a tradeoff, they can do many things in a dynamic way that compiled 
programs may not do easily or at all. 

Another argument I find is unfair, is some comparisons with what I consider 
"mixed" effort. If you have a program that calls various numpy routines using 
native python in between, then clearly a decent amount of the time spent is not 
in numpy. It may for example suddenly import a module and that takes time spent 
doing nothing about the problem other than loading what is needed. Another 
example, it is common to place the numbers or other values you get from numpy 
operations into dictionaries, lists and so on, or to make graphs. If you had 
done the same work in a purely C (or FORTRAN or whatever environment) and had 
access to similar other functionality, the latter two would all be in C or some 
compiled library. 

With exceptions aplenty, speed is not always a primary consideration. Often 
programmer time and productivity matter more. I have seen many projects though 
that were first implemented in a language like Python and when they had figured 
out a decent way to do the task reliably, they handed it over to people to redo 
much or all of it using languages like C++, compiles and all, as a reasonable 
translation from a working application may take much less time to implement and 
once done, way work better. At least it may until you want to change it! 
Prototyping often works better in some languages. Python has a compromise that 
modules can use to speed up important parts of the process by substituting a 
compiled function in C/C++/whatever for a local Python function but not 
necessarily switching entirely to C and suffering under the negatives of that 
environment.

I do wonder if we are already on a path where a language that handles concepts 
like parallelism (or even weirder, quantum computations) well may be a better 
candidate for doing some projects well as even though it may not run at top 
speed, it can make use of many processors or a network of machines in the 
cloud, to handle things in a more flexible way that may even get the job done 
faster.

Next discussion is whether pandas is faster than C, followed by SciPy ...

I do have to wonder if anyone ever considered adding back enough functionality 
into base Python to make some additions less needed. Is there any reason the 
kind of structures used by so many languages cannot be made part of python such 
as a vector/array that holds exactly one kind of data structure and not force 
use of things like a list when that is more than is needed?


-Original Message-
From: Grant Edwards 
To: python-list@python.org
Sent: Fri, Feb 25, 2022 4:12 pm
Subject: Re: C is it always faster than nump?


On 2022-02-25, Richard Damon  wrote:

> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:

>> Hi,

>> a lot of people think that C (or C++) is faster than python, yes I agree,

>> but I think that's not the case with numpy, I believe numpy is faster than

>> C, at least in some cases.

>

> My understanding is that numpy is written in C,



Sort of. IIRC a lot of the heavly lifting is done by libraries like

BLAS and LAPAK that were written in FORTRAN the last time I checked.



Has that changed?



Or am I conflating numpy with some other scientific-python stuff. Back

when I did a lot of numerical stuff in Python, I remember spending a

lot of time watching FORTRAN compiles. Admittedly, that's getting to

be 10+ years ago...



--

Grant





-- 

https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: One-liner to merge lists?

[Albert-Jan Roskam]

 If you don't like the idea of 'adding' strings you can 'concat'enate:

 >>> items = [[1,2,3], [4,5], [6]]
 >>> functools.reduce(operator.concat, items)
 [1, 2, 3, 4, 5, 6]
 >>> functools.reduce(operator.iconcat, items, [])
 [1, 2, 3, 4, 5, 6]

 The latter is the functional way to spell your for... extend() loop.
 Don't forget to provide the initial value in that case lest you modify
 the input:

 >> functools.reduce(operator.iconcat, items)  # wrong
 [1, 2, 3, 4, 5, 6]
 >>> items
 [[1, 2, 3, 4, 5, 6], [4, 5], [6]]  # oops

 --
 https://mail.python.org/mailman/listinfo/python-list

   
   Was also thinking about reduce, though this one uses a dunder method:
   from functools import reduce
   d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
   print(reduce(list.__add__, list(d.values(
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Chris Angelico]

On Sat, 26 Feb 2022 at 09:58, Richard Damon  wrote:
>
> On 2/25/22 2:47 PM, Chris Angelico wrote:
> > On Sat, 26 Feb 2022 at 05:49, Richard Damon  
> > wrote:
> >> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> >>> Hi,
> >>> a lot of people think that C (or C++) is faster than python, yes I agree,
> >>> but I think that's not the case with numpy, I believe numpy is faster than
> >>> C, at least in some cases.
> >>>
> >> My understanding is that numpy is written in C, so for it to be faster
> >> than C, you are saying that C is faster that C.
> > Fortran actually, but ultimately, they both get compiled to machine code.
>
> Looking at the Github repo I see:
>
> Languages:
> Python.  62.5%
> C   35.3%
> C++.   1.0%
> Cython.   0.9%
> Shell.   0.2%
> Fortran.   0.1%
>
> So there is a bit of Fortan in there, but it looks like most of the
> heavy lifting is in C.
>
> My guess is the Fortran is likely some hooks to add Fortran modules into
> the program with numpy.
>

GitHub's analysis isn't always very meaningful. I think it's pretty
clear that Numpy isn't implemented in Python :) In any case, point is
that the implementation language is mostly irrelevant.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Richard Damon]

On 2/25/22 2:47 PM, Chris Angelico wrote:

On Sat, 26 Feb 2022 at 05:49, Richard Damon  wrote:

On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:

Hi,
a lot of people think that C (or C++) is faster than python, yes I agree,
but I think that's not the case with numpy, I believe numpy is faster than
C, at least in some cases.


My understanding is that numpy is written in C, so for it to be faster
than C, you are saying that C is faster that C.

Fortran actually, but ultimately, they both get compiled to machine code.


Looking at the Github repo I see:

Languages:
Python.  62.5%
C       35.3%
C++.   1.0%
Cython.   0.9%
Shell.   0.2%
Fortran.   0.1%

So there is a bit of Fortan in there, but it looks like most of the 
heavy lifting is in C.


My guess is the Fortran is likely some hooks to add Fortran modules into 
the program with numpy.


...

The key point is that numpy was written by skilled programmers who
carefully optimized their code to be as fast as possible for the major
cases. Thus it is quite possible for the numpy code to be faster in C
than code written by a person without that level of care and effort.

This is clearly true.

ChrisA



--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

[issue46845] dict: Use smaller entry for Unicode-key only dict.

[Inada Naoki]

Inada Naoki  added the comment:

>
>
> Do you propose to
> 1. Only use StringKeyDicts when non-string keys are not possible?  (Where
> would this be?)
> 2. Switch to a normal dict when a non-string key is added?  (But likely
> not switch back when the last non-string key is removed.)
> 3. Deprecate and remove the option to add non-string keys to namespace
> dicts?  (Proposed and rejected at least once as not gaining much.)
>
>
>

2. We already do such hack for key sharing dict.
And yes, deleting non string key doesn't switch back. d[0]=0; del d[0];
loop must be amortized O(1).
Only dict.clear() switches back.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46838] Parameters and arguments parser syntax error improvments

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +29713
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31590

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[Chris Angelico]

On Sat, 26 Feb 2022 at 09:04, Barry Scott  wrote:
>
>
>
> > On 25 Feb 2022, at 18:07, Abdur-Rahmaan Janhangeer  
> > wrote:
> >
> > Normally people put Python in the scripting category.
> > I learnt typed languages like C++ and Java at first.
> > People who learn languages like these tend to put
> > Python in a non-serious category. The post was
> > more ironic than litteral.
>
> I think that python is a serious language, as I do C and C++.
> What is key to a problem is which language is best to solve the
> problem (given you have a free choice).
>

Exactly. What language is best for a situation depends on myriad factors:

* Development convenience
* Execution performance
* Language/library tools
* Interaction with other parts of a system
* Corporate politics
* The ability to use other people's code
* Etcetera, etcetera, etcetera.

What's the best language for modding Kerbal Space Program? C#, because
ultimately, anything else will require a C# wrapper.

What's the best language for building a scientific simulation of
particle collisions? Probably something high level like Python, backed
by some hard number-crunching work in Fortran.

What's the best language for writing a browser-based game? JavaScript.

What's the best language for swearing in? Ah, that one I can't help
you with, although I've heard good things about French. :)

The only languages that are "not serious languages" in that sense are
deliberate toys. Chef, Piet, Whitespace, Ook, these are non-serious
languages that you shouldn't be writing production code in. Anything
else? If it's right for your purposes, use it, and don't let anyone
shame you for it.

There are almost no "right" and "wrong" design decisions. Your
decision has consequences, and if you accept those consequences, the
decision was not wrong.

Dismissing a mainstream language as "not serious" is, in effect,
telling everyone who uses that language "you're doing it wrong". And
that's an insult that should not be made lightly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to check if there is internet?

[Barry Scott]

> On 25 Feb 2022, at 18:07, Abdur-Rahmaan Janhangeer  
> wrote:
> 
> Normally people put Python in the scripting category.
> I learnt typed languages like C++ and Java at first.
> People who learn languages like these tend to put
> Python in a non-serious category. The post was
> more ironic than litteral.

I think that python is a serious language, as I do C and C++.
What is key to a problem is which language is best to solve the
problem (given you have a free choice).

My day job is writing and maintaining a large python application.
Python allows us to use best engineering technics.

Python is used because it out performs C/C++/etc in the speed that
we can implement new features and generate revenue from them.

> 
> After 5 years of following the mailing lists i realised
> that there is more than the eyes meet. My hobby has
> always been language engineering / compiler theory
> and just recently i realised that Python is pretty serious about
> language features.
> 
> I have been following language feature proposals from various
> languages. Some decide to avoid Python's route,
> but others have been trying hard to catch up with Python.
> One gleaming example is the switch case. JS recently proposed pattern
> matching, referencing Python and explaining why the proposal
> is a cool treatment of the usecase.
> 
> As a side note, if by scripting we mean OS commands,
> then Python started as a sysadmin language.

It was started as a research language to teach children programming.
Python's simplicity, that we benefit from to this day, comes from that original 
purpose.

And indeed one of the pleasures of the UK PYCON is seeing children present
their projects to a room full of adult programers. Seeing a 6 year old present 
their
working project is wonderful to behold.

Barry


> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46856] datetime.max conversion

[Eric V. Smith]

Eric V. Smith  added the comment:

Probably so. You could step through the code to make sure that's what's going 
on.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46856] datetime.max conversion

[Joris Geysens]

Joris Geysens  added the comment:

I see this in the python source code being tested (datetimetester.py), so I 
guess it is a rounding problem : 

# maximum timestamp: set seconds to zero to avoid rounding issues
max_dt = self.theclass.max.replace(tzinfo=timezone.utc,
   second=0, microsecond=0)
max_ts = max_dt.timestamp()
# date -12-31 23:59:00+00:00: timestamp 253402300740
self.assertEqual(self.theclass.fromtimestamp(max_ts, tz=timezone.utc),
 max_dt)

--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[Michael F. Stemper]

On 25/02/2022 14.30, 2qdxy4rzwzuui...@potatochowder.com wrote:

On 2022-02-25 at 13:48:32 -0600,
"Michael F. Stemper"  wrote:


On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:



I have been following language feature proposals from various
languages. Some decide to avoid Python's route, but others have been
trying hard to catch up with Python.  One gleaming example is the
switch case. JS recently proposed pattern matching, referencing
Python and explaining why the proposal is a cool treatment of the
usecase.



I'm not clear on what you mean here. JavaScript has had a switch/case
construct since 1.2, in the late 1990s. As far as I can determine,
python has no such thing, since PEP-3103 was rejected in 2007.


Python has a relatively new (as of version 3.10) "match" statement:

 https://docs.python.org/3/reference/compound_stmts.html#the-match-statement


Looks as if I have some reading to do. Thanks.


--
Michael F. Stemper
Always use apostrophe's and "quotation marks" properly.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to check if there is internet?

[Chris Angelico]

On Sat, 26 Feb 2022 at 07:32, <2qdxy4rzwzuui...@potatochowder.com> wrote:
>
> On 2022-02-25 at 13:48:32 -0600,
> "Michael F. Stemper"  wrote:
>
> > On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:
>
> > > I have been following language feature proposals from various
> > > languages. Some decide to avoid Python's route, but others have been
> > > trying hard to catch up with Python.  One gleaming example is the
> > > switch case. JS recently proposed pattern matching, referencing
> > > Python and explaining why the proposal is a cool treatment of the
> > > usecase.
>
> > I'm not clear on what you mean here. JavaScript has had a switch/case
> > construct since 1.2, in the late 1990s. As far as I can determine,
> > python has no such thing, since PEP-3103 was rejected in 2007.
>
> Python has a relatively new (as of version 3.10) "match" statement:
>
> 
> https://docs.python.org/3/reference/compound_stmts.html#the-match-statement

Every language learns from every other. Python has learned from
JavaScript too. Both languages have learned from Haskell. Pike has
learned from Python; and Python has learned from Pike. Nearly every
language learns from C and Smalltalk (sometimes indirectly). Of
course, not everything that is learned is copied - sometimes what you
learn is "let's not do it that way".

Python's match statement is not a switch/case block, and if JS is
borrowing the idea, then that's strong evidence that, even with
switch/case, match/case is a valuable addition. They behave very
differently in usage.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46854] Failed to compile static python3.7.12

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46849] Memory problems detected using Valgrind

[Terry J. Reedy]

Terry J. Reedy  added the comment:

It is possible that some core developer may get regular valgrind reports, but I 
would not know who.  I am pretty sure that you should run it at least on the 
latest release of the 'main' branch, 3.11.0a5, and perhaps even better, the 
current tip.  There might be a performance patch that was not backported.  (I 
have no idea what iotop is.)

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46855] printing a string with strange characters loops forever

[svilen dobrev]

svilen dobrev  added the comment:

aha. ctrl-s also closes it.
seems kind-of ( ctrl-q - ctrl-s )
 https://en.wikipedia.org/wiki/Software_flow_control

thanks, closing this.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Grant Edwards]

On 2022-02-25, Richard Damon  wrote:
> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
>> Hi,
>> a lot of people think that C (or C++) is faster than python, yes I agree,
>> but I think that's not the case with numpy, I believe numpy is faster than
>> C, at least in some cases.
>
> My understanding is that numpy is written in C,

Sort of. IIRC a lot of the heavly lifting is done by libraries like
BLAS and LAPAK that were written in FORTRAN the last time I checked.

Has that changed?

Or am I conflating numpy with some other scientific-python stuff. Back
when I did a lot of numerical stuff in Python, I remember spending a
lot of time watching FORTRAN compiles. Admittedly, that's getting to
be 10+ years ago...

--
Grant


-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46845] dict: Use smaller entry for Unicode-key only dict.

[Terry J. Reedy]

Terry J. Reedy  added the comment:

CPython, at least, allows users to insert non-string keys in namespace dicts 
that are conceptually string-key only.

>>> globals()[0] = 'zero'
>>> globals()[0]
'zero'
>>> vars()
{'__name__': '__main__', ..., 0: 'zero'}

[This is for consenting adults only, as it prevents sorting keys and 
string-only operations on keys.
>>> dir()
...
TypeError: '<' not supported between instances of 'int' and 'str']
 
Do you propose to
1. Only use StringKeyDicts when non-string keys are not possible?  (Where would 
this be?)
2. Switch to a normal dict when a non-string key is added?  (But likely not 
switch back when the last non-string key is removed.)
3. Deprecate and remove the option to add non-string keys to namespace dicts?  
(Proposed and rejected at least once as not gaining much.)

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46858] mmap constructor resets the file pointer on Windows

[benrg]

New submission from benrg :

On Windows, `mmap.mmap(f.fileno(), ...)` has the undocumented side effect of 
setting f's file pointer to 0.

The responsible code in mmapmodule is this:

/* Win9x appears to need us seeked to zero */
lseek(fileno, 0, SEEK_SET);

Win9x is no longer supported, and I'm quite sure that NT doesn't have whatever 
problem they were trying to fix. I think this code should be deleted, and a 
regression test added to verify that mmap leaves the file pointer alone on all 
platforms.

(mmap also maintains its own file pointer, the `pos` field of `mmap_object`, 
which is initially set to zero. This issue is about the kernel file pointer, 
not mmap's pointer.)

--
components: IO, Library (Lib), Windows
messages: 414039
nosy: benrg, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: mmap constructor resets the file pointer on Windows
type: behavior
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46838] Parameters and arguments parser syntax error improvments

[Terry J. Reedy]

Terry J. Reedy  added the comment:

With two exceptions, nice suggestions if feasible.

>>> def foo(*args=None): pass
SyntaxError: * argument cannot have default value

>>> def foo(**kwargs=None): pass
SyntaxError: ** argument cannot have default value

Good.

>>> foo(*args=[0])
SyntaxError: cannot assign to iterable argument unpacking

>>> foo(**args={"a": None})
SyntaxError: cannot assign to keyword argument unpacking

Incomprehensible.  It seems to me that these should have same message as first 
two; message should not depend on proposed default.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46833] Windows installer is unclear and has redundant settings

[Terry J. Reedy]

Change by Terry J. Reedy :


--
title: Installer Wizard is unclear and has redundant settings -> Windows 
installer is unclear and has redundant settings

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[2QdxY4RzWzUUiLuE]

On 2022-02-25 at 13:48:32 -0600,
"Michael F. Stemper"  wrote:

> On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:

> > I have been following language feature proposals from various
> > languages. Some decide to avoid Python's route, but others have been
> > trying hard to catch up with Python.  One gleaming example is the
> > switch case. JS recently proposed pattern matching, referencing
> > Python and explaining why the proposal is a cool treatment of the
> > usecase.

> I'm not clear on what you mean here. JavaScript has had a switch/case
> construct since 1.2, in the late 1990s. As far as I can determine,
> python has no such thing, since PEP-3103 was rejected in 2007.

Python has a relatively new (as of version 3.10) "match" statement:

https://docs.python.org/3/reference/compound_stmts.html#the-match-statement
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46816] Remove declarations for non-__STDC__ compilers

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46810] Make multiprocessing.connection.Client support ipv6

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +davin, pitrou
title: multiprocessing.connection.Client doesn't support ipv6 -> Make 
multiprocessing.connection.Client support ipv6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Best way to check if there is internet?

[Michael F. Stemper]

On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:


I have been following language feature proposals from various
languages. Some decide to avoid Python's route,
but others have been trying hard to catch up with Python.
One gleaming example is the switch case. JS recently proposed pattern
matching, referencing Python and explaining why the proposal
is a cool treatment of the usecase.


I'm not clear on what you mean here. JavaScript has had a
switch/case construct since 1.2, in the late 1990s. As far
as I can determine, python has no such thing, since PEP-3103
was rejected in 2007.


As a side note, if by scripting we mean OS commands,
then Python started as a sysadmin language.


I never knew this. Where can I read more about this origin?

--
Michael F. Stemper
If it isn't running programs and it isn't fusing atoms, it's just bending space.
--
https://mail.python.org/mailman/listinfo/python-list

[issue37426] getpass.getpass not working with on windows when ctrl+v is used to enter the string

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -terry.reedy
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Chris Angelico]

On Sat, 26 Feb 2022 at 06:44, Avi Gross via Python-list
 wrote:
>
> I agree with Richard.
>
> Some people may be confused and think c is the speed of light and 
> relativistically speaking, nothing can be faster. (OK, just joking. The uses 
> of the same letter of the alphabet are not at all related. One is named for 
> the language that came after the one named B, while the other may be short 
> for celeritas meaning speed.)
>
> There is no such thing as C. C does nothing. It is a combination of a 
> language specification and some pieces of software called compilers that 
> implement it well or less well.
>

Uhh, that's taking it a little bit TOO far I agree with your
point, but saying that there's no such thing as C is slightly unfair
:)

> There is such a thing as a PROGRAM. A program completely written in C is a 
> thing. It can run fast or slow based on a combination of how it was written 
> and on what data it operates on, which hardware and OS and so on. AND some of 
> it may likely be running code from libraries written in other languages like 
> FORTRAN that get linked into it in some way at compile time or runtime, and 
> hooks into the local OS and so on.
>
> So your program written supposedly in pure C, may run faster or slower. If 
> you program a "sort" algorithm in C, it may matter if it is an implementation 
> of a merge sort or at bubble sort or ...
>

More specifically: You're benchmarking a particular *implementation*
of a particular *algorithm*. Depending on what you're trying to
demonstrate, either could be significant.

Performance testing between two things written in C is a huge job.
Performance testing across languages has a strong tendency to be
meaningless (like benchmarking Python's integers against JavaScript's
numbers).

> As noted, numpy is largely written in C. It may well be optimized in some 
> places but there are constraints that may well make it hard to optimize 
> compared to some other implementation without those constraints. In 
> particular, it interfaces with standard Python data structures at times such 
> as when initializing from a Python List, or List of Lists, or needing to hold 
> on to various attributes so it can be converted back, or things I am not even 
> aware of.
>

(Fortran)

In theory, summing a Numpy array should be incredibly fast, but in
practice, there's a lot of variation, and it can be quite surprising.
For instance, integers are faster than floats, everyone knows that.
And it's definitely faster to sum smaller integers than larger ones.

rosuav@sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(100), dtype=numpy.float64)' 'numpy.sum(x)'
1000 loops, best of 5: 325 usec per loop
rosuav@sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(100), dtype=numpy.int64)' 'numpy.sum(x)'
500 loops, best of 5: 551 usec per loop
rosuav@sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(100), dtype=numpy.int32)' 'numpy.sum(x)'
500 loops, best of 5: 680 usec per loop

... Or not.

Summing arrays isn't necessarily the best test of numpy anyway, but as
you can see, testing is an incredibly difficult thing to get right.
The easiest thing to prove is that you have no idea how to prove
anything usefully, and most of us achieve that every time :)

ChrisA


> So, I suspect it may well be possible to make a pure C library similar to 
> numpy in many ways but that can only be used within a C program that only 
> uses native C data structures. It also is possible to write such a program 
> that is horribly slow. And it is possible to write a less complex version of 
> numpy that does not support some current numpy functionality and overall runs 
> much faster on what it does support.
>
> I do wonder at the reason numpy and pandas and lots of other modules have to 
> exist. Other languages like R made design choices that built in ideas of 
> vectorization from the start. Python has lots of object-oriented 
> extensibility that can allow you to create interpreted code that may easily 
> extend it in areas to have some similar features. You can create an 
> array-like data structure that holds only one object type and is extended so 
> adding two together (or multiplying) ends up doing it componentwise. But 
> attempts to do some such things often run into problems as they tend to be 
> slow. So numpy was not written in python, mostly, albeit it could have been 
> even more impressive if it took advantage of more pythonic abilities, at a 
> cost.
>
> But now that numpy is in C, pretty much, it is somewhat locked in when and if 
> other things in Python change.
>
> The reality is that many paradigms carried too far end up falling short.
>
>
> -Original Message-
> From: Richard Damon 
> To: python-list@python.org
> Sent: Fri, Feb 25, 2022 1:48 pm
> Subject: Re: C is it always faster than nump?
>
>
> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> > Hi,
> > a lot of people think that C (or 

Re: C is it always faster than nump?

[Chris Angelico]

On Sat, 26 Feb 2022 at 05:49, Richard Damon  wrote:
>
> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> > Hi,
> > a lot of people think that C (or C++) is faster than python, yes I agree,
> > but I think that's not the case with numpy, I believe numpy is faster than
> > C, at least in some cases.
> >
> My understanding is that numpy is written in C, so for it to be faster
> than C, you are saying that C is faster that C.

Fortran actually, but ultimately, they both get compiled to machine code.

Really, what the OP has demonstrated is that good, well-written code
called from bad code produces meaningless numbers that are not the
same as bad code written in C. I can't even use that to prove that
good code is faster than bad code, since the measurements aren't
comparable; but if that were what was being measured, it wouldn't be
very surprising.

To do a fair comparison of C and Numpy, you'd have to:

1) Use the same type of timer
2) Use the same algorithm (unless you're benchmarking "naive C code"
against "well-written Fortran code")
3) Have comparable levels of compile-time optimization
4) Cope with the vagaries of CPU caching
5) Ensure that the same data types are being used everywhere
6) Probably several other things that I didn't think of.

The precise way you run the test could easily skew it by orders of
magnitude in either direction.

> The key point is that numpy was written by skilled programmers who
> carefully optimized their code to be as fast as possible for the major
> cases. Thus it is quite possible for the numpy code to be faster in C
> than code written by a person without that level of care and effort.

This is clearly true.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Avi Gross via Python-list]

I agree with Richard.

Some people may be confused and think c is the speed of light and 
relativistically speaking, nothing can be faster. (OK, just joking. The uses of 
the same letter of the alphabet are not at all related. One is named for the 
language that came after the one named B, while the other may be short for 
celeritas meaning speed.)

There is no such thing as C. C does nothing. It is a combination of a language 
specification and some pieces of software called compilers that implement it 
well or less well.

There is such a thing as a PROGRAM. A program completely written in C is a 
thing. It can run fast or slow based on a combination of how it was written and 
on what data it operates on, which hardware and OS and so on. AND some of it 
may likely be running code from libraries written in other languages like 
FORTRAN that get linked into it in some way at compile time or runtime, and 
hooks into the local OS and so on.

So your program written supposedly in pure C, may run faster or slower. If you 
program a "sort" algorithm in C, it may matter if it is an implementation of a 
merge sort or at bubble sort or ...

As noted, numpy is largely written in C. It may well be optimized in some 
places but there are constraints that may well make it hard to optimize 
compared to some other implementation without those constraints. In particular, 
it interfaces with standard Python data structures at times such as when 
initializing from a Python List, or List of Lists, or needing to hold on to 
various attributes so it can be converted back, or things I am not even aware 
of.

So, I suspect it may well be possible to make a pure C library similar to numpy 
in many ways but that can only be used within a C program that only uses native 
C data structures. It also is possible to write such a program that is horribly 
slow. And it is possible to write a less complex version of numpy that does not 
support some current numpy functionality and overall runs much faster on what 
it does support.

I do wonder at the reason numpy and pandas and lots of other modules have to 
exist. Other languages like R made design choices that built in ideas of 
vectorization from the start. Python has lots of object-oriented extensibility 
that can allow you to create interpreted code that may easily extend it in 
areas to have some similar features. You can create an array-like data 
structure that holds only one object type and is extended so adding two 
together (or multiplying) ends up doing it componentwise. But attempts to do 
some such things often run into problems as they tend to be slow. So numpy was 
not written in python, mostly, albeit it could have been even more impressive 
if it took advantage of more pythonic abilities, at a cost.

But now that numpy is in C, pretty much, it is somewhat locked in when and if 
other things in Python change. 

The reality is that many paradigms carried too far end up falling short.


-Original Message-
From: Richard Damon 
To: python-list@python.org
Sent: Fri, Feb 25, 2022 1:48 pm
Subject: Re: C is it always faster than nump?


On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> Hi,
> a lot of people think that C (or C++) is faster than python, yes I agree,
> but I think that's not the case with numpy, I believe numpy is faster than
> C, at least in some cases.
>
My understanding is that numpy is written in C, so for it to be faster 
than C, you are saying that C is faster that C.

The key point is that numpy was written by skilled programmers who 
carefully optimized their code to be as fast as possible for the major 
cases. Thus it is quite possible for the numpy code to be faster in C 
than code written by a person without that level of care and effort.

There are similar package available for many languages, including C/C++ 
to let mere mortals get efficient numerical processing.

-- 
Richard Damon


-- 
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Chris Angelico]

On Sat, 26 Feb 2022 at 03:13, BELAHCENE Abdelkader
 wrote:
> *This is the Python3 program :import timeit as itimport numpy as npimport
> systry : n=eval(sys.argv[1])except: print ("needs integer as argument") ;
> exit() a=range(1,n+1)b=np.array(a)def func1(): return sum(a)def
> func2(): return np.sum(b)print(f"sum with Python: {func1()} and NumPy
> {func2()} ")tm1=it.timeit(stmt=func1, number=n)print(f"time used Python
> Sum: {round(tm1,2)} sec")tm2=it.timeit(stmt=func2, number=n)print(f"time
> used  Numpy Sum: {round(tm2,2)} sec")*

This is terrible code. Even aside from the messed-up formatting (for
which your mail client is probably to blame), using eval() and a bare
"except:" clause is not a good way to use Python. And then you get
timeit to do as many iterations as the length of the array, which is
hardly indicative or meaningful for small values.

> *and Here the C program:#include #include #include
> long func1(int n){ long  r=0;for (int  i=1; i<=
> n;i++) r+= i; return r;}int main(int argc, char* argv[]){
> clock_t c0, c1; long v,count; int n;   if ( argc < 2)
> {  printf("Please give an argument"); return
> -1;  }n=atoi(argv[1]); c0 = clock();*
>
>
>
>
> *   for (int j=0;j < n;j++) v=func1(n); c1 = clock(); printf
> ("\tCPU  time :%.2f sec", (float)(c1 - c0)/CLOCKS_PER_SEC);
> printf("\n\tThe value : %ld\n",  v);}*

At least you're consistent, using an iteration count equal to the
length of the array again. But that just means that it's equally
meaningless.

Did you know that Python's timeit and C's clock don't even measure the
same thing?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

Short summary:

PersistentTaskGroup shares the followings from TaskGroup:
- It uses WeakSet to keep track of child tasks.
- After exiting the async context manager scope (or the shutdown procedure), it 
ensures that all tasks are complete or cancelled.

PersistentTaskGroup differs in that:
- It keeps running after all tasks successfully finish unless it is explicitly 
shutdown or the parent task is cancelled.
- It is one of the main use cases that shutdown() method is called separately.  
The shutdown procedure may be triggered from different task contexts.
- It provides two-phase cancellation with a configurable grace period.
- It does not propagate unhandled exceptions and cancellations from child tasks 
to the outside of the task group and sibling tasks but calls a customizable 
fallback exception handler. -> This could be done without modifying TaskGroup.

The API looks similar to TaskGroup with minor modification.
The semantics of a PersistentTaskGroup more resembles a nested event loop, in 
that it has its own set of tasks, it keeps running until closed, and it has its 
own fallback exception handler.

Note that current aiotools implementation lacks many details, such as two-phase 
cancellation.  I'm going to implement more soon.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Threading question .. am I doing this right?

[Chris Angelico]

On Sat, 26 Feb 2022 at 05:16, Robert Latest via Python-list
 wrote:
>
> Chris Angelico wrote:
> > Depending on your database, this might be counter-productive. A
> > PostgreSQL database running on localhost, for instance, has its own
> > caching, and data transfers between two apps running on the same
> > computer can be pretty fast. The complexity you add in order to do
> > your own caching might be giving you negligible benefit, or even a
> > penalty. I would strongly recommend benchmarking the naive "keep going
> > back to the database" approach first, as a baseline, and only testing
> > these alternatives when you've confirmed that the database really is a
> > bottleneck.
>
> "Depending on your database" is the key phrase. This is not "my" database that
> is running on localhost. It is an external MSSQL server that I have no control
> over and whose requests frequently time out.
>

Okay, cool. That's crucial to know.

I'm still curious as to the workload (requests per second), as it
might still be worth going for the feeder model. But if your current
system works, then it may be simplest to debug that rather than
change.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to check if there is internet?

[Chris Angelico]

On Sat, 26 Feb 2022 at 05:09, Abdur-Rahmaan Janhangeer
 wrote:
>
> Normally people put Python in the scripting category.

You have a very interesting definition of "normal".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46853] Python interpreter can get code from memory, it is not secure.

[Brett Cannon]

Brett Cannon  added the comment:

This is by design and thus not a security hole or bug.

Also, for any future security issues, please disclose them responsibly by 
following the instructions at https://www.python.org/dev/security/ .

--
nosy: +brett.cannon
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26175] Fully implement IOBase abstract on SpooledTemporaryFile

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Also, which of the two patches.

Irit, you just patched Temp file doc, can you look at the PR code?

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46843] PersistentTaskGroup API

[Joongi Kim]

Joongi Kim  added the comment:

> And just a question: I'm just curious about what happens if belonging tasks 
> see the cancellation raised from their inner tasks.  Sibling tasks should not 
> be cancelled, and the outer task group should not be cancelled, unless the 
> task group itself has requested cancellation.  Could the new cancellation 
> counter help this?

To achieve this by distinguishing cancellation from inner/outer tasks, 
TaskGroup._on_task_done() should be modified to skip setting _on_completed_fut 
because it should keep running.  Swallowing exceptions in child tasks can be 
done without modifying TaskGroup, but this part requires changes of TaskGroup.

Another difference is the usage.  Instead of relying on the async context 
manager interface, we would call "TaskGroup.shutdown()" separately from either 
directly in signal handlers or from cleanup methods of long-lived objects that 
have task groups as attributes.

And I also want to perform two-phase cancellation: instead of cancelling all 
tasks immediately as in current _abort(), have a configurable grace period 
until they have chances to complete and then cancel with additional timeout on 
cancellation itself to prevent hangs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26175] Fully implement IOBase abstract on SpooledTemporaryFile

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Éric, you might use git log or git blame to see who that is active has patched 
the relevant file in the last few years.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: C is it always faster than nump?

[Richard Damon]

On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:

Hi,
a lot of people think that C (or C++) is faster than python, yes I agree,
but I think that's not the case with numpy, I believe numpy is faster than
C, at least in some cases.

My understanding is that numpy is written in C, so for it to be faster 
than C, you are saying that C is faster that C.


The key point is that numpy was written by skilled programmers who 
carefully optimized their code to be as fast as possible for the major 
cases. Thus it is quite possible for the numpy code to be faster in C 
than code written by a person without that level of care and effort.


There are similar package available for many languages, including C/C++ 
to let mere mortals get efficient numerical processing.


--
Richard Damon

--
https://mail.python.org/mailman/listinfo/python-list

[issue46856] datetime.max conversion

[Joris Geysens]

Joris Geysens  added the comment:

a ValueError is raised : 

ValueError: year 1 is out of range 

on 

dt = datetime.fromtimestamp(ts, tz=timezone.utc)

or 

dt = datetime.utcfromtimestamp(ts)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com