[issue46929] __rrshift__ for same class obj will raise TypeError

2022-03-05 Thread ZhengYu, Xu


ZhengYu, Xu  added the comment:

But with different classed will be right

```
class C:
def __rrshift__(self, v):
return 1

class D:
def __rrshift__(self, v):
return 2

C() >> D() # 2
```

--

___
Python tracker 
<https://bugs.python.org/issue46929>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46929] __rrshift__ for same class obj will raise TypeError

2022-03-05 Thread ZhengYu, Xu


New submission from ZhengYu, Xu :

```
class C:
def __rrshift__(self, v):
return 1

C() >> C()   # raise TypeError: unsupported operand types(s) for >>: 'C' and 'C'
```

--
messages: 414578
nosy: zen-xu
priority: normal
severity: normal
status: open
title: __rrshift__ for same class obj will  raise TypeError
type: behavior
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue46929>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46055] Speed up binary shifting operators

2021-12-12 Thread Xinhang Xu


Xinhang Xu  added the comment:

I post a comment to the PR showing its performance improvement. I paste it 
below. I think the result not bad.

-

I use timeit to measure time costs and any other operators or calls are 
excluded. For each testcase, the former is dcd2796 and the latter is this PR's 
base 036bbb1.

64-bit Release building. Run in Windows 10 1709 (64-bit)

python -m timeit " i = 1; i <<= 3; i >>= 3"  # small value (cost down by 36%)
500 loops, best of 5: 92.7 nsec per loop
200 loops, best of 5: 145 nsec per loop

python -m timeit " i = 1; i <<= 10; i >>= 10"  # medium value (-25%)
200 loops, best of 5: 114 nsec per loop
200 loops, best of 5: 151 nsec per loop

python -m timeit " i = 1; i <<= 100; i >>= 100"  # big value  (-12%)
100 loops, best of 5: 209 nsec per loop
100 loops, best of 5: 238 nsec per loop

--

___
Python tracker 
<https://bugs.python.org/issue46055>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46055] Speed up binary shifting operators

2021-12-11 Thread Xinhang Xu


New submission from Xinhang Xu :

See its PR.

-

Inspired by [bpo-44946](https://bugs.python.org/issue44946), I found there were 
no special shortcuts for shifting operation applied to "medium value" long 
object. So I modified CPython's VM to accelerate my python project where there 
is plenty of binary shifting operation. I guess somebody else might also need 
it.

--
components: Interpreter Core
messages: 408362
nosy: xuxinhang
priority: normal
pull_requests: 28289
severity: normal
status: open
title: Speed up binary shifting operators
type: performance
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46055>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44946] Integer operations are inefficient for "medium" integers.

2021-12-10 Thread Xinhang Xu


Change by Xinhang Xu :


--
nosy: +xuxinhang
nosy_count: 3.0 -> 4.0
pull_requests: +28270
pull_request: https://github.com/python/cpython/pull/30044

___
Python tracker 
<https://bugs.python.org/issue44946>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45880] Performance regression of Int object operators. (Python 3.11)

2021-11-23 Thread Xinhang Xu


New submission from Xinhang Xu :

Hello.

I'm working on a compute-bound project recently, so I tested several Python 
versions on my PC (Windows 10 64-bit), about Python's performance on operating 
Int object. All Python binaries are official distributions.


Testcase #1 (simple xor op)

Source:

import cProfile as profile
profile.run('for _ in range(5): 5 ^ 6')

The given result:

   C:\Users\surface\Desktop\PythonTest>python-3.9.9-embed-amd64\python 
C:\Users\surface\python_test.py
 3 function calls in 24.398 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1   24.398   24.398   24.398   24.398 :1()
10.0000.000   24.398   24.398 {built-in method builtins.exec}
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.10.0-embed-amd64\python 
C:\Users\surface\python_test.py
 3 function calls in 27.941 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1   27.941   27.941   27.941   27.941 :1()
10.0000.000   27.941   27.941 {built-in method builtins.exec}
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.11.0a2-embed-amd64\python 
C:\Users\surface\python_test.py
 3 function calls in 42.209 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1   42.209   42.209   42.209   42.209 :1()
10.0000.000   42.209   42.209 {built-in method builtins.exec}
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects}



Testcase #2 (simple add op)

Source:

import cProfile as profile
profile.run('for _ in range(5): 5 + 6')

The given result:

   C:\Users\surface\Desktop\PythonTest>python-3.9.9-embed-amd64\python 
C:\Users\surface\python_test.py
 3 function calls in 24.599 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1   24.599   24.599   24.599   24.599 :1()
10.0000.000   24.599   24.599 {built-in method builtins.exec}
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.10.0-embed-amd64\python 
C:\Users\surface\python_test.py
 3 function calls in 27.414 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1   27.414   27.414   27.414   27.414 :1()
10.0000.000   27.414   27.414 {built-in method builtins.exec}
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects}



   C:\Users\surface\Desktop\PythonTest>python-3.11.0a2-embed-amd64\python 
C:\Users\surface\python_test.py
 3 function calls in 43.675 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
1   43.675   43.675   43.675   43.675 :1()
10.0000.000   43.675   43.675 {built-in method builtins.exec}
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects}



As you can see, Python 3.11 costs *much more* time to execute Int object 
operator. I have also tested the same cases on another Windows PC, the result 
shows the same.

Is it a common thing? What's the reason for this problem?


Thanks.

--
messages: 406849
nosy: xuxinhang
priority: normal
severity: normal
status: open
title: Performance regression of Int object operators. (Python 3.11)
type: performance
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45880>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35540] dataclasses.asdict breaks with defaultdict fields

2021-10-28 Thread Raymond Xu


Raymond Xu  added the comment:

I am seeing this bug with 3.9.7

--
nosy: +greenfish6

___
Python tracker 
<https://bugs.python.org/issue35540>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44022] "urllib" will result to deny of service

2021-05-05 Thread Gen Xu


Gen Xu  added the comment:

Looks like it is caused by the httplib not limiting total header size after 
receiving 100. Added a counter for that to be same size as _MAXLINE=65536.

--
versions:  -Python 3.7

___
Python tracker 
<https://bugs.python.org/issue44022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44022] "urllib" will result to deny of service

2021-05-05 Thread Gen Xu


Gen Xu  added the comment:

Added a possible PR. Review will be appreicated.

--

___
Python tracker 
<https://bugs.python.org/issue44022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44022] "urllib" will result to deny of service

2021-05-05 Thread Gen Xu


Change by Gen Xu :


--
keywords: +patch
nosy: +gen-xu
nosy_count: 1.0 -> 2.0
pull_requests: +24585
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/25916

___
Python tracker 
<https://bugs.python.org/issue44022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43140] built-in open() doesn't use locale.getpreferredencoding() as the default encoding

2021-02-05 Thread Cake Xu


Cake Xu  added the comment:

Thank Eryk for answering my question.

So I get it now. I use this in Linux.

If my understanding is right, the open() will invoke 
locale.getpreferredencoding() by setting the do_setlocale=False -- i.e. 
locale.getpreferredencoding(False) -- to avoid invoking setlocale(LC_CTYPE, "").

Previously I had thought it's locale.getpreferredencoding(True) that invoked.

--

___
Python tracker 
<https://bugs.python.org/issue43140>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43140] build-in open() doesn't use whatever locale.getpreferredencoding() returns as default encoding.

2021-02-05 Thread Cake Xu


Change by Cake Xu :


--
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue43140>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43140] build-in open() doesn't use whatever locale.getpreferredencoding() returns as default encoding.

2021-02-05 Thread Cake Xu


New submission from Cake Xu :

In the document about build-in function open() 
https://docs.python.org/3/library/functions.html#open, it says that
"encoding is the name of the encoding used to decode or encode the file. This 
should only be used in text mode. The default encoding is platform dependent 
(whatever locale.getpreferredencoding() returns), but any text encoding 
supported by Python can be used. See the codecs module for the list of 
supported encodings.
"
But as I tried, I found that after I set locale using locale.setlocale(), the 
default encoding used by open() was changed and thus sometimes led to a 
UnicodeDecodeError.

So I think that the default encoding used by open() is the second element of 
whatever locale returned by locale.getlocale().

--
assignee: docs@python
components: Documentation
messages: 386544
nosy: docs@python, smallbigcake
priority: normal
severity: normal
status: open
title: build-in open() doesn't use whatever locale.getpreferredencoding() 
returns as default encoding.
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43140>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42930] xml.parsers.expat results differ buffer_text and / or buffer_size

2021-01-15 Thread Michael XU


Michael XU  added the comment:

Thank you so much Walter. I think that might be it - it fixed this particular 
instance, and it makes sense given what you have said. I'll proceed to close 
this up but will follow up if I encounter this issue when the data has changed.

Thanks again!

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42930>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42930] xml.parsers.expat results differ buffer_text and / or buffer_size

2021-01-14 Thread Michael XU


New submission from Michael XU :

More details available here:
https://stackoverflow.com/questions/65676934/python-xml-parsers-expat-results-differ-based-on-buffer-text-true-or-false-and

Raw data to replicate the issue is available upon request.

--
components: XML
messages: 385086
nosy: mfxuus
priority: normal
severity: normal
status: open
title: xml.parsers.expat results differ buffer_text and / or buffer_size
type: behavior
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue42930>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42872] Inconsistent exceptions thrown by pathlib.Path.mkdir on different OSes

2021-01-09 Thread Hong Xu


Hong Xu  added the comment:

Should we update the document at least? The document doesn't mention 
NotADirectoryError or its super classes at all.

--

___
Python tracker 
<https://bugs.python.org/issue42872>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42872] Inconsistent exceptions thrown by pathlib.Path.mkdir on different OSes

2021-01-09 Thread Hong Xu


Change by Hong Xu :


--
title: Inconsistent exceptions thrown by mkdir on different OSes -> 
Inconsistent exceptions thrown by pathlib.Path.mkdir on different OSes

___
Python tracker 
<https://bugs.python.org/issue42872>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42872] Inconsistent exceptions thrown by mkdir on different OSes

2021-01-09 Thread Hong Xu


Change by Hong Xu :


--
title: Inconsistent exception thrown by mkdir on different OSes -> Inconsistent 
exceptions thrown by mkdir on different OSes

___
Python tracker 
<https://bugs.python.org/issue42872>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42872] Inconsistent exception thrown by mkdir on different OSes

2021-01-09 Thread Hong Xu


New submission from Hong Xu :

Consider the following code:

-

import pathlib

def main():
pathlib.Path('tmp').touch()
pathlib.Path('tmp/tmp_sub').mkdir(parents=True)

main()



Run the code above in an empty directory.

On Linux, it throws a `NotADirectory` exception:

Traceback (most recent call last):
  File "", line 1, in 
  File "main.py", line 5, in main
pathlib.Path('tmp/tmp_sub').mkdir(parents=True)
  File "/usr/lib/python3.8/pathlib.py", line 1287, in mkdir
self._accessor.mkdir(self, mode)
NotADirectoryError: [Errno 20] Not a directory: 'tmp/tmp_sub'

-

On Windows, it throws a FileExistsError exception:

Traceback (most recent call last):
  File "C:\Users\hong\anaconda3\lib\pathlib.py", line 1284, in mkdir
self._accessor.mkdir(self, mode)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 
'tmp\\tmp_sub'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 7, in 
main()
  File "main.py", line 5, in main
pathlib.Path('tmp/tmp_sub').mkdir(parents=True)
  File "C:\Users\hong\anaconda3\lib\pathlib.py", line 1288, in mkdir
self.parent.mkdir(parents=True, exist_ok=True)
  File "C:\Users\hong\anaconda3\lib\pathlib.py", line 1284, in mkdir
self._accessor.mkdir(self, mode)
FileExistsError: [WinError 183] Cannot create a file when that file already 
exists: 'tmp

--
components: Library (Lib), Windows
messages: 384715
nosy: Hong Xu, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Inconsistent exception thrown by mkdir on different OSes
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42872>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39241] Popen of python3.6 hangs on os.read(errpipe_read, 50000)

2020-01-10 Thread Xu


Xu  added the comment:

Thanks Stinner.

Do you have some clues about this issue ?
In my case, our test system use Popen to start a few infrastructure processes 
for tests going. After tests done we will kill all processes and start again 
for new round.

Most of time it works fine.  However we have some chances where one Popen hangs 
on "os.read(errpipe_read, 5) forever.

The close_fd is always true.

I guess there is some subtle issue here.

--

___
Python tracker 
<https://bugs.python.org/issue39241>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39241] Popen of python3.6 hangs on os.read(errpipe_read, 50000)

2020-01-06 Thread Xu


New submission from Xu :

I have a piece code hangs on os.read(errpipe_read, 5)
So I compared the python3.6 with python2.7 on _execute_child, I saw:

for python2.7 we create the errpipe_read/write with pipe_cloexec()

1213 # For transferring possible exec failure from child to parent
1214 # The first char specifies the exception type: 0 means
1215 # OSError, 1 means some other error.
1216 errpipe_read, errpipe_write = self.pipe_cloexec()


while for python3.6 we create the errpipe_read/write with pipe()

1251 # For transferring possible exec failure from child to parent.
1252 # Data format: "exception name:hex errno:description"
1253 # Pickle is not used; it is complex and involves memory 
allocation.
1254 errpipe_read, errpipe_write = os.pipe()

Does that mean python3.6 doesn't set the the flag FD_CLOEXEC on the pipe ?

--
components: Library (Lib)
messages: 359486
nosy: liuxu1005
priority: normal
severity: normal
status: open
title: Popen of python3.6 hangs on os.read(errpipe_read, 5)
versions: Python 3.6

___
Python tracker 
<https://bugs.python.org/issue39241>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24935] LDSHARED is not set according when CC is set.

2019-07-25 Thread Hong Xu


Change by Hong Xu :


--
pull_requests: +14719
pull_request: https://github.com/python/cpython/pull/14950

___
Python tracker 
<https://bugs.python.org/issue24935>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37659] subprocess.list2cmdline() should not escape wrapping single/double quotes

2019-07-23 Thread James Xu


New submission from James Xu :

While working on our project, we have noticed that for 
`subprocess.Popen(command, ...)`, when `command` is a string that contains 
escaped double quote, for example, `command = '"path to executable" --flag 
arg'`, this works fine. However, when command is changed to 
`shlex.split(command, posix=False)`, the Popen command fails. Looking a bit 
into the source code, it seems that for the command above,

```
>>> shlex.split('"path to executable" --flag arg', posix=False)
['"path to executable"', '--flag', 'arg']
```

and when this array of strings gets passed into `Popen`, the escaped double 
quote gets escaped again, since `subprocess.list2cmdline` does not check if a 
pair of double quote or single quote are wrapping the entire string `arg`. And 
this is the same behavior for both py2 and py3, 
https://github.com/python/cpython/blob/master/Lib/subprocess.py#L508. As a 
result, upon execution the command becomes, `'"\\"path to executable\\"" --flag 
arg'`

example:
```
>>> sp.list2cmdline(['"do things"'])
'"\\"do things\\""'
>>> sp.list2cmdline(['do things'])
'"do things"'
>
```

--
components: Windows
messages: 348342
nosy: kejxu, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: subprocess.list2cmdline() should not escape wrapping single/double quotes
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37659>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37572] email lib bug

2019-07-13 Thread famu xu

famu xu  added the comment:

import smtplib
from email.message import EmailMessage
from email.utils import formataddr

server = smtplib.SMTP('smtp.xxx.com',port=25)
server.login('y...@xxx.com', 'password')
msg = EmailMessage()

#if address username include Chinese or other multibytes language, it will be 
encode to utf8 and base64, like '?utf-8?B?6YKu5Lu25rWL6K+V?= ', 
but the raw smtp request data error, "From: xxx" and "To: xxx" fields has the 
error end char:"\r\r\r\r\n".
#From: ?utf-8?B?6YKu5Lu25rWL6K+V?= \r\r\r\r\n

msg['From'] =formataddr(('中文','y...@xxx.com'))
msg['To'] = formataddr(('中文姓名','t...@yyy.com'))

msg['Subject'] =subject
msg.set_content('something')

server.send_message(msg)
server.quit()

--

___
Python tracker 
<https://bugs.python.org/issue37572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37572] email lib bug

2019-07-12 Thread famu xu


famu xu  added the comment:

python 3.7 same

--

___
Python tracker 
<https://bugs.python.org/issue37572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37572] email lib bug

2019-07-12 Thread famu xu


New submission from famu xu :

file: email\_header_value_parser.py

line : encoded_part = part.fold(policy=policy)[:-1] # strip nl
modify to: encoded_part = part.fold(policy=policy)[:-2] # strip nl

because the nl is "\r\n"

--
components: email
messages: 347740
nosy: barry, famu xu, r.david.murray
priority: normal
severity: normal
status: open
title: email lib bug
versions: Python 3.6

___
Python tracker 
<https://bugs.python.org/issue37572>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37225] Signatures of Exceptions not documented

2019-06-11 Thread Hong Xu


Hong Xu  added the comment:

Thanks for your answer, but I believe this is a real document bug. OSError does 
have its signature documented, but the majority of other exception classes do 
not do so, neither does BaseException explains a default behavior clearly (see 
my quote above).

As an example, ValueError accepts multiple arguments and makes use of all of 
them when given, but from the document, I can barely guess this out.

--

___
Python tracker 
<https://bugs.python.org/issue37225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37225] Signatures of Exceptions not documented

2019-06-11 Thread Hong Xu


New submission from Hong Xu :

The "Builtin Exceptions" page does not document the constructors of the listed 
exception classes. All it says is

> The tuple of arguments given to the exception constructor. Some built-in 
> exceptions (like OSError) expect a certain number of arguments and assign a 
> special meaning to the elements of this tuple, while others are usually 
> called only with a single string giving an error message.

This is quite vague and does not really guide users for individual exception 
classes.

--
assignee: docs@python
components: Documentation
messages: 345195
nosy: Hong Xu, docs@python
priority: normal
severity: normal
status: open
title: Signatures of Exceptions not documented
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue37225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32382] Python mulitiprocessing.Queue fail to get according to correct sequence

2017-12-20 Thread Weize Xu

New submission from Weize Xu <vet@gmail.com>:

I try to implement a "producer consumer" like design with mulitiprocessing 
module in my project, but I found that mulitiprocessing.Queue 's behavior is 
not as my expected. It seems Queue.get method return the end flag at the end of 
my queue too early. 

I am not experienced at muliti-process programing, I am not sure it's a bug or 
not. For reproduce this, I have simplified my code as following:

``` Python
import time
import multiprocessing as mp

def worker(task_queue, output_queue):
while 1:
i = task_queue.get()
if i is None:
print("Process-%d done"%mp.current_process().pid)
task_queue.task_done()
break
output_queue.put(i+1)
task_queue.task_done()

def outputer(output_queue):
c = 0 # val for count how many obj geted
while 1:
j = output_queue.get()
if j is None:
print("Process(output)-%d done"%mp.current_process().pid)
c += 1
print("outputer get %d objects from the output_queue"%c)
assert output_queue.empty(), "output queue should be empty here"
break
time.sleep(0.0001) # do output here
c += 1

if __name__ == "__main__":
task_queue = mp.JoinableQueue()
#output_queue = mp.JoinableQueue()
output_queue = mp.Queue()

workers = [mp.Process(target=worker, args=(task_queue, output_queue))
   for i in range(10)]
outputer = mp.Process(target=outputer, args=(output_queue,))

for w in workers:
w.start()
outputer.start()

for i in range(10**6):
task_queue.put(i)
for w in workers: # put end flag to task queue
task_queue.put(None)

task_queue.join() # wait all tasks done
print("all tasks done.")

print("queue size before put end flag: %d"%output_queue.qsize())
output_queue.put(None) # put end flag to output queue

print("end")
```

Get the output:

Process-20923 done
Process-20931 done
Process-20925 done
Process-20930 done
Process-20927 done
Process-20929 done
Process-20928 done
Process-20926 done
Process-20924 done
Process-20932 done
all tasks done.
queue size before put end flag: 914789
end
Process(output)-20933 done
outputer get 90383 objects from the output_queue
Process Process-11:
Traceback (most recent call last):
  File "/home/nanguage/S/miniconda3/lib/python3.6/multiprocessing/process.py", 
line 249, in _bootstrap
self.run()
  File "/home/nanguage/S/miniconda3/lib/python3.6/multiprocessing/process.py", 
line 93, in run
self._target(*self._args, **self._kwargs)
  File "joinablequeue.py", line 27, in outputer
assert output_queue.empty(), "output queue should be empty here"
AssertionError: output queue should be empty here


I have wait all worker put the output to the output queue use 
taks_queue.join(), then I put the end flag to the output queue, but according 
to outputer's printed information, it get the `None` end flag before other 
value in the queue. It seems queue not get value according to 'FIFO' rule.

--
components: Library (Lib)
messages: 308710
nosy: Weize Xu, davin
priority: normal
severity: normal
status: open
title: Python mulitiprocessing.Queue fail to get according to correct sequence
type: behavior
versions: Python 3.6

___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32382>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31255] Test getrandom before using it

2017-08-22 Thread Benda Xu

Benda Xu added the comment:

Thanks Victor. Python-3.5 has received your patches in time and 3.5.4 builds 
well with glibc-2.25 and https://wiki.gentoo.org/wiki/Project:Prefix
  2. 
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6c8e2396d9cf4fe13057741f5bd1b9989eeeb746

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31255] Test getrandom before using it

2017-08-22 Thread Benda Xu

Changes by Benda Xu <hero...@gmail.com>:


--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31255] Test getrandom before using it

2017-08-22 Thread Benda Xu

Benda Xu added the comment:

> Does it only affect Python 3.4 and 3.5? Both versions are in security 
> fix-only mode and no longer receive other fixes.

Sorry I haven't noticed that there are two new releases this month Python 3.4.7 
and 3.5.4.  The test build results are:

3.4.7 fail
3.4.6 fail
3.5.4 success
3.5.3 fail
3.6.1 success

I will remove the python 3.5 version tag.  Looks like Python 3.5 got this 
compatibility fix in time before entering security update phase.  But python 
3.4 did not, making it incompatible with glibc-2.25 on old linux kernels.

--
versions:  -Python 3.5

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31255] Test getrandom before using it

2017-08-22 Thread Benda Xu

Benda Xu added the comment:

This is a continuation of http://bugs.python.org/issue29157#msg288291.

--
nosy: +Vladimír Čunát, christian.heimes
versions: +Python 3.5

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31255] Test getrandom before using it

2017-08-22 Thread Benda Xu

Changes by Benda Xu <hero...@gmail.com>:


--
versions:  -Python 3.5, Python 3.6, Python 3.7

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31255] Test getrandom before using it

2017-08-22 Thread Benda Xu

Changes by Benda Xu <hero...@gmail.com>:


--
components: +Build

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31255] Test getrandom before using it

2017-08-22 Thread Benda Xu

Changes by Benda Xu <hero...@gmail.com>:


--
nosy: +haypo

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue18050] embedded interpreter or virtualenv fails with ImportError: cannot import name MAXREPEAT

2013-06-01 Thread H Xu

Changes by H Xu h...@topbug.net:


--
nosy: +xuhdev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue18050
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14489] repr() function link on the built-in function documentation is incorrect

2012-07-21 Thread H Xu

H Xu xuh...@gmail.com added the comment:

I think this has been fixed. Where did you see the incorrect link?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14489
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14489] repr() function link on the built-in function documentation is incorrect

2012-04-04 Thread H Xu

New submission from H Xu xuh...@gmail.com:

The `repr()` built-in function link  in this page [ 
http://docs.python.org/library/functions.html ] should link to the built-in 
version of `repr()`.
It should link to: http://docs.python.org/library/functions.html#repr

However, it links to here: http://docs.python.org/library/repr.html#module-repr

--
assignee: docs@python
components: Documentation
messages: 157462
nosy: H.Xu, docs@python
priority: normal
severity: normal
status: open
title: repr() function link on the built-in function documentation is incorrect
versions: Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14489
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue11106] python 2.6.6 and python 2.7.1 cannot be built successfully because of an segment fault on NetBSD-5.1-sparc

2011-02-03 Thread H Xu

New submission from H Xu xuh...@gmail.com:

Build python 2.6.6 and python 2.7.1 on a NetBSD-5.1-sparc machine.

1. Run './configure';
2. Run 'make';
3. Run 'make install'.

There will be a problem after run 'make install'.
The last few lines of error messages are like the following:

Compiling /usr/local/lib/python2.6/test/test_binop.py ...
Compiling /usr/local/lib/python2.6/test/test_bisect.py ...
Compiling /usr/local/lib/python2.6/test/test_bool.py ...
Compiling /usr/local/lib/python2.6/test/test_bsddb.py ...
Compiling /usr/local/lib/python2.6/test/test_bsddb185.py ...
Compiling /usr/local/lib/python2.6/test/test_bsddb3.py ...
Compiling /usr/local/lib/python2.6/test/test_buffer.py ...
Compiling /usr/local/lib/python2.6/test/test_bufio.py ...
Compiling /usr/local/lib/python2.6/test/test_builtin.py ...
[1]   Segmentation fault (core dumped) PYTHONPATH=/usr/...
*** Error code 139

Stop.
make: stopped in /home/xuh/src/Python-2.6.6

Same thing with python 2.7.1.

--
components: Build
messages: 127802
nosy: H.Xu
priority: normal
severity: normal
status: open
title: python 2.6.6 and python 2.7.1 cannot be built successfully because of an 
segment fault on NetBSD-5.1-sparc
type: compile error
versions: Python 2.6, Python 2.7

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11106
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue11106] python 2.6.6 and python 2.7.1 cannot be built successfully because of an segment fault on NetBSD-5.1-sparc

2011-02-03 Thread H Xu

H Xu xuh...@gmail.com added the comment:

The result of 'make install SHELL=bash -x' seems nothing different with the 
make install. Could there be any other way to debug?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11106
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com