[issue26195] Windows frozen .exe multiprocessing.Queue access is denied exception
Alex Robinson added the comment: Sorry I can't help more than provide a test environment for any fix. I just plucked the "fix" from StackOverflow and it fixed the Q problem on my machine. It appears, at the least, the multiprocessing code should probably not rely on the default value for the 'inheritable' argument. That argument does sound like one that might be different in the usual case for Win32 and Unix, just because. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26195> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26195] Windows frozen .exe multiprocessing.Queue access is denied exception
New submission from Alex Robinson: A pyinstaller 3.0 frozen .exe Python 2.7.10 program under Windows 7 that uses a multiprocessing.Queue to send things to a multiprocessing.Process leads to the process getting access-is-denied exceptions on every q.get() call. And, when the program can't exit. Or leaves a dangling process for every Process. An unsatisfying fix for this is to put the following code somewhere in the program: """ Do what must be done to make multiprocessing work in .exe files. This involves monkey patching multiprocessing.forking under Windows so when the a program using a multiprocessing.Process exits, there won't be processes left running. Hint from http://stackoverflow.com/questions/33764448/pathos-multiprocessing-pipe-and-queue-on-windows . The bottom line is we get "access is denied" when trying . to get from the multiprocessing.Queue when we're an .exe file. . So, in multiprocessing.forking.duplicate(), . we change 'inheritable' to default to True . from the normal code's False. """ import sys import multiprocessing import multiprocessing.forking # # # # def duplicate(handle, target_process = None, inheritable = True) : return(multiprocessing.forking.kludge_to_fix_dangling_processes(handle, target_process, inheritable)) if (not hasattr(multiprocessing.forking, 'kludge_to_fix_dangling_processes')) and (sys.platform == 'win32') : multiprocessing.forking.kludge_to_fix_dangling_processes= multiprocessing.forking.duplicate multiprocessing.forking.duplicate = duplicate -- components: Library (Lib) messages: 258895 nosy: alex_python_org priority: normal severity: normal status: open title: Windows frozen .exe multiprocessing.Queue access is denied exception type: behavior versions: Python 2.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26195> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py: add writesamples() and readsamples()
Alex Robinson alex_python_...@tranzoa.com added the comment: Here go, Terry. Copies of the two files in the latest ZIP file. Hmmm. Well. Maybe just one of 'em. Looks like the only way to upload files is to add a msg, so I'll upload the other file in another msg. -- Added file: http://bugs.python.org/file18451/wave_futz.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4913 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py: add writesamples() and readsamples()
Alex Robinson alex_python_...@tranzoa.com added the comment: OK, here's the other. -- Added file: http://bugs.python.org/file18452/test_wave.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4913 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py: add writesamples() and readsamples()
Alex Robinson alex_python_...@tranzoa.com added the comment: I'll upload the latest monkey-patch file, wave_futz.py, and test_wave.py, which has a gob of tests added to it. I found a 64-bit bug in the wave.py formats for 32-bit sample wave files. The pcm files read in to CoolEdit ok, including the 32-bit sample files. Added file: http://bugs.python.org/file12985/wave_futz.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4913 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py: add writesamples() and readsamples()
Alex Robinson alex_python_...@tranzoa.com added the comment: Polo: I could do it, but I'm in disagreement with big part of your patch. Why surely you can't mean the bug. :) (The test program has it fixed.) What is the disagreement? Apparently this bug system allows file attachments, so I will upload a test program and wave file. The program is hard coded to read the wave file and write a bunch of wave files, the names of which describe what they sound like. Added file: http://bugs.python.org/file12703/wave_futz.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4913 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py: add writesamples() and readsamples()
Alex Robinson alex_python_...@tranzoa.com added the comment:
8 bit samples stored as unsigned bytes?
8 bit samples are 0..255 in the file. But to work with them, you'll want
them -128..127. The code assumes DC==0 sample values for simplicity.
if len(wavs) not in [ 1, 2, 4 ] ?
That way if you're working with mono, you can simply pass your samples
down to writesamples() without having to remember to [ samples ] them.
If you forget, no big deal. It's too bad that readsamples() can't know
that you want only mono samples. That would make mono work simpler.
Anyway, I don't argue very strongly for this spiff. In some ways it's
worse to be there. After all, the caller may be writing 1 frame at a
time, though I don't think that such logic would work. And would be
pretty slow, too, probably.
Calling setnchannels?
Since the number of channels *must* be the number of sample arrays
passed to writesamples(), either writesamples() must rely on the caller
already having gratuitously set the number of channels (correctly), or
writesamples() can simply force the issue. If the caller set the number
of channels wrongly, then the output file will be corrupt or
writesamples() would need to raise an exception. Both just make work for
the caller. get/set_nchannels() are not particularly useful if you are
using the read/write_sample API. If there were no ..frame() API,
getnchannels() might still be handy to use to find out how many channels
a wave file being read has before any samples have been read. But that's
about it.
integers, is that the best format?
Far bettern than the byte stream form which is useless, confusing, error
prone, and exposes the internal wave file format to the caller who could
generally care less how a wave file stores the samples. But, you bring
up a very good point, I think. I forgot to int(s) the samples when they
are put in to the arrays. The reason for an int() call on all the
samples is so that the caller can deal with samples as floats. (Which is
how he will want to deal with them if he's doing anything interesting.)
So, this:
ws = array.array('l', [ int(s) for s in wav ])
ws = array.array('h', [ int(s) for s in wav ])
ws = array.array('B', [ int(s + 128) for s in wav ])
And, for testing, in normalize_samples():
samps = [ s * mxm for s in samps ]
So normalize_samples() always sends floats to writesamples().
The code layout?
:) Well. Whatever. I know that the official python thing is to push
colons left. I don't like that. I've experimented in the last few years
with doing a lot of vertical alignment. Over time, I've found that it is
a great way to do things. It's pretty amazing how much easier it is to
scan and read code that has, as a start, the ='s vertically aligned.
And, over the last few years, I've put more and more blank lines in.
Vertically compressed code tends to look like assembler language. I edit
with 200+ character wide screen so, since I stopped forcing everything
to fit on TTY, text mode CGA, or punch cards, I've lost a taste for
narrow code. In fact, I personally have a real hard time reading
line-broken code. That said, multi-lining if statements in ways that
allow delete/insert of lines, 1 for each operator () expression, can
be very nice. Calls to routines with a gob of parameters, each on a
separate line, can sure be a good way to deal with a bad thing (routines
that take a lot of params are bad, that is). Etc. Anyway, I assumed that
the code would be reformatted by whomever maintains wave.py. No biggy.
hand write the wave file for testing?
Good point! Allows a test to do things like odd numbers of frames, 1
frame, max'ed out sample values, long runs of silence, DC offsets, etc.
Do you know whether there are already test files for wave.py? They'd
have those sorts of things in them. Hmmm. It's odd that wave.py doesn't
run from the command line and dump the header or something, at least.
Maybe do some simple conversions (8/16, mono/stereo switches, reverses
... that sort of thing). Would be handy.
This code is not tested on a big-endian machine. I ran it under XP
(py2.4) and Ubuntu64 (py2.5) and all the output files CRC the same on
both PCs.
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4913
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py: add writesamples() and readsamples()
Alex Robinson alex_python_...@tranzoa.com added the comment: DC (0 hz) assumption? wave.py makes the assumption that what the user wants is whatever happens to be in the file, however arbitrary. (That 8 bit samples are unsigned bytes is probably an artifact of early ADC logic. Typically you got an absolute, n-bit value from an old ADC. Newer chips often return signed values.) It's very unlikely that anything but a copy program would try to work with unsigned char samples. Too many things to go wrong. Too much confusion. Zero means zero in most of the world, in and out of audio processing. :) That said, not having to offset the 8 bit samples sparsifies the read/write_sample code. But, I'm thinking that that's at the expense of every program that uses it. When in doubt, I figure, do what is more useful. Don't force the caller to write a wrapper if he'll need to do it 99% of the time. But this is not a religious thing with me. A wrapper can be written. And, in fact, I'd sure think it would be nice to include wrappers like auto-scaling and auto-zeroing in wave.py. But maybe not, as these ops probably belong in some array.py type module. Anyway, a non-audio guy who just wants to read a wave file, diddle with it, and write it out. Or who just wants to generate some sound and write it out. Or who just wants to read a wave file and graph it or something. All of these guys will be stunned when they find out to their hours-of-work chagrin that wave files' 8 bit samples are not signed chars. And, if I were one of them, I'd be plenty peeved after having to spend all that time learning about some historical artifact just to read an danged audio file, for gosh sakes. But not putting the 8-bit offset in the read/write_samples logic does eliminate 2 lines of code in each routine. writesamples would raise an exception Yep, taste. I'm inclined to find this irritating and I don't like being irritated by packages I use. Makes for a poor out-of-box experience. But, taste. :) 4) Well, lets fix a format then. else: wavs = [[wav]] ? That's an extra [] I think. [[samples]] would be an array of array of an array of samples. s = [1,2]; print [ [ a ] ];[[[1,2]]] On reflection, I'd say I agree with you more than I do with me on the ability of writesamples() to take a simple array of mono samples. Not a good thing to do. wavs.extend(wav) ? I had to look up extend() and try it in the Python shell! :) To each his own. But when I found out that one could do list+=added_list in Python I never looked back. Intuitive. I special-cased mono for speed purposes. No reason to do the +=/extend for mono samples. But, maybe the interpretor handles all that. Don't know. Didn't measure it. monkey patch?' Wonderful! This makes your rewrite of the code *so* much cleaner. Thanks for the tip! code layout? Har, har. Yep, no one in software has ever spent any time discussing code layout before. Let's do it for the first time in history. test_wave.py? Oooo. Bit minimal, that. Yeah, I think a couple of things could be fleshed out there. Gotta run now. But will try to update the code in wave_futz later. Other things on plate, though. Guilherme, I really appreciate your handling this and your guidance. Thanks! ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4913 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py writes 16 bit sample files of half the correct duration
Alex Robinson alex_python_...@tranzoa.com added the comment:
Oh golly. I was confused. For some reason I was thinking
writesamples() when using writeframes().
So the current code reads ok. Which makes this bug a request for
writesamples() and readsamples() to be added to wave.py. They would
shield sleep deprived saps from the .wav file data frame format. :)
Here are python2.4-ish versions written for outside wave.py. Combos of 8
and 16 bit samples, mone and stereo, are tested. I did not test the
32-bit sample logic.
Sample values are expected to be +-32767 or +-128 ints (or +-2.x gig if
32-bit).
def readsamples(wf, nframes) :
Read an array of number-of-channels normalized int sample
arrays.
wav = wf.readframes(nframes)
ifwf.getsampwidth() == 4 :
wav = struct.unpack(%ul % (len(wav) / 4), wav)
elif wf.getsampwidth() == 2 :
wav = struct.unpack(%uh % (len(wav) / 2), wav)
else :
wav = struct.unpack(%uB % len(wav), wav)
wav = [ s - 128 for s in wav ]
nc = wf.getnchannels()
if nc 1 :
wavs= []
for i in xrange(nc) :
wavs.append([ wav[si] for si in xrange(0, len(wav), nc) ])
pass
else :
wavs= [ wav ]
return(wavs)
def writesamples(wf, wavs) :
Write samples to the wave file.
'wavs' looks like this:
[ left_channel_samples, right_channel_samples ]
or [ left_channel_samples ]
or mono_samples
This routine calls setnchannels() from information about 'wavs'
length.
if wavs :
if len(wavs) not in [ 1, 2, 4 ] :
wavs= [ wavs, wv ]
wf.setnchannels(len(wavs))
if len(wavs)1 :
wav = []
for w in zip(*wavs):
wav+= w
pass
else :
wav = wavs[0]
ifwf.getsampwidth() == 4 :
ws = array.array('l', [ s for s in wav ])
elif wf.getsampwidth() == 2 :
ws = array.array('h', [ s for s in wav ])
else :
ws = array.array('B', [ s + 128 for s in wav ])
ws = ws.tostring()
wf.writeframes(ws)
pass
# end of code to edit and insert in wave.py
--
type: behavior - feature request
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4913
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py writes 16 bit sample files of half the correct duration
Alex Robinson alex_python_...@tranzoa.com added the comment: Oh gob. I left a debug artifact in that code. wavs= [ wavs, wv ] needs to be without the 'wv'. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4913 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4913] wave.py writes 16 bit sample files of half the correct duration
New submission from Alex Robinson alex_python_...@tranzoa.com: Corrected code in writeframesraw(): self._datawritten = self._datawritten + len(data) * self._sampwidth else: self._file.write(data) self._datawritten = self._datawritten + len(data) * self._sampwidth Note that the default (not byte swapped) assignment to _datawritten must also be multiplied by _sampwidth. If not, audio programs will ignore the second half of a 16-bit-sample file. As a side note, the calls to _patchheader() do not need to be protected by this if statement: if self._datalength != self._datawritten: _patchheader does the same test to optimize its operation. -- components: Library (Lib) messages: 79586 nosy: alex_python_org severity: normal status: open title: wave.py writes 16 bit sample files of half the correct duration type: behavior versions: Python 2.4, Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4913 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2754] Mac version of IDLE doesn't scroll as expected
New submission from Alex Robinson [EMAIL PROTECTED]: On a Mac running 10.5.2 in IDLE 1.2.1, the scroll bars don't move as expected track pad/mouse scrolling is not enabled. I believe this is also the case on Tiger (10.4) When using the scroll bar to scroll up through code, the scroll bar moves much faster than the mouse pointer. When scrolling down, the mouse pointer moves slightly faster. Unlike Terminal, scrolling using a scroll wheel on a mouse or the two fingers on the trackpad doesn't work with IDLE. I realize these are fairly minor UI issues, but they're still frustrating and slow down my workflow. -- components: IDLE, Macintosh messages: 66202 nosy: alextrob severity: normal status: open title: Mac version of IDLE doesn't scroll as expected type: behavior versions: Python 2.5 __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue2754 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
