[issue7946] Convoy effect with I/O bound threads and New GIL

2021-01-01 Thread David Beazley
Change by David Beazley : -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue7946> ___ ___ Python-bugs-list

[issue16894] Function attribute access doesn't invoke methods in dict subclasses

2021-01-01 Thread David Beazley
Change by David Beazley : -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue16894> ___ ___ Python-bugs-list

[issue24844] Python 3.5rc1 compilation error with Apple clang 4.2 included with Xcode 4

2021-01-01 Thread David Beazley
Change by David Beazley : -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue24844> ___ ___ Python-bugs-list

[issue32810] Expose ags_gen and agt_gen in asynchronous generators

2021-01-01 Thread David Beazley
Change by David Beazley : -- stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue32810> ___ ___ Pyth

[issue27436] Strange code in selectors.KqueueSelector

2021-01-01 Thread David Beazley
Change by David Beazley : -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue27436> ___ ___ Python-bugs-list

[issue16132] ctypes incorrectly encodes .format attribute of memory views

2021-01-01 Thread David Beazley
Change by David Beazley : -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue16132> ___ ___ Python-bugs-list

[issue7946] Convoy effect with I/O bound threads and New GIL

2020-10-03 Thread David Beazley
David Beazley added the comment: About nine years ago, I stood in front of a room of Python developers, including many core developers, and gave a talk about the problem described in this issue. It included some live demos and discussion of a possible fix. https://www.youtube.com/watch?v

[issue33014] Clarify doc string for str.isidentifier()

2018-03-10 Thread David Beazley
David Beazley added the comment: s = 'Some String' s.isalnum() s.isalpha() s.isdecimal() s.isdigit() s.isidentifier() s.islower() s.isnumeric() s.isprintable() s.isspace() s.istitle() s.isupper() Not really sure where I would have gotten the idea that it might be referring to s

[issue33014] Clarify doc string for str.isidentifier()

2018-03-09 Thread David Beazley
David Beazley added the comment: That wording isn't much better in my opinion. If I'm sitting there looking at methods like str.isdigit(), str.isnumeric(), str.isascii(), and str.isidentifier(), seeing keyword.iskeyword() makes me think it's a method regardless of whether

[issue33014] Clarify doc string for str.isidentifier()

2018-03-06 Thread David Beazley
New submission from David Beazley : This is a minor nit, but the doc string for str.isidentifier() states: Use keyword.iskeyword() to test for reserved identifiers such as "def" and "class". At first glance, I thought that it meant you'd do this (doesn't wor

[issue32810] Expose ags_gen and agt_gen in asynchronous generators

2018-02-10 Thread David Beazley
David Beazley added the comment: I've attached a file that illustrates the issue. (Side thought: this would be nice to have in inspect or traceback) -- Added file: https://bugs.python.org/file47434/agen.py ___ Python tracker &

[issue32810] Expose ags_gen and agt_gen in asynchronous generators

2018-02-09 Thread David Beazley
New submission from David Beazley : Libraries such as Curio and asyncio provide a debugging facility that allows someone to view the call stack of generators/coroutines. For example, the _task_get_stack() function in asyncio/base_tasks.py. This works by manually walking up the chain of

[issue32690] Return function locals() in order of creation?

2018-01-28 Thread David Beazley
David Beazley added the comment: Some context: I noticed this while discussing (in a course) a programming trick involving instance initialization and locals() that I'd encountered in the past: def _init(locs): self = locs.pop('self') for name, va

[issue27436] Strange code in selectors.KqueueSelector

2016-07-01 Thread David Beazley
David Beazley added the comment: I don't see any possible way that you would ever get events = EVENT_READ | EVENT_WRITE if the flag is a single value (e.g., KQ_FILTER_READ) and the flag itself is not a bitmask. Only one of those == tests will ever be True. There is no need t

[issue27436] Strange code in selectors.KqueueSelector

2016-07-01 Thread David Beazley
David Beazley added the comment: If the KQ_FILTER constants aren't bitmasks, it seems that the code could be simplified to the last version then. At the least, it would remove a few unnecessary calculations.Again, a very minor thing (I only stumbled onto it by accident r

[issue27436] Strange code in selectors.KqueueSelector

2016-07-01 Thread David Beazley
New submission from David Beazley: Not so much a bug, but an observation based on reviewing the implementation of the selectors.KqueueSelector class. In that class there is the select() method: def select(self, timeout=None): timeout = None if timeout is None else max

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

2015-10-26 Thread David Beazley
David Beazley added the comment: Please don't make flush() close the file on a BlockingIOError. That would be an unfortunate mistake and make it impossible to implement non-blocking I/O correctly with buffered I/O. -- ___ Python tracker

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

2015-10-25 Thread David Beazley
New submission from David Beazley: First comment: In the I/O library, there is documented behavior for how things work in the presence of non-blocking I/O. For example, read/write methods returning None on raw file objects. Methods on BufferedIO instances raise a BlockingIOError for

[issue7322] Socket timeout can cause file-like readline() method to lose data

2015-10-25 Thread David Beazley
David Beazley added the comment: This bug is still present in Python 3.5, but it occurs if you attempt to do a readline() on a socket that's in non-blocking mode. In that case, you probably DO want to retry at a later time (unlike the timeout

[issue24975] Python 3.5 can't compile AST involving PEP 448 unpacking

2015-08-31 Thread David Beazley
New submission from David Beazley: The compile() function is not able to compile an AST created from code that uses some of the new unpacking generalizations in PEP 448. Example: code = ''' a = { 'x'

[issue24844] Python 3.5rc1 compilation error on OS X 10.8

2015-08-11 Thread David Beazley
New submission from David Beazley: Just a note that Python-3.5.0rc1 fails to compile on Mac OS X 10.8.5 with the following compiler: bash$ clang --version Apple LLVM version 4.2 (clang-425.0.28) (based on LLVM 3.2svn) Target: x86_64-apple-darwin12.6.0 Thread model: posix bash$ Here is the

[issue23441] rlcompleter: tab on empty prefix => insert spaces

2015-07-26 Thread David Beazley
David Beazley added the comment: It's still broken on Python 3.5b4. -- ___ Python tracker <http://bugs.python.org/issue23441> ___ ___ Python-bugs-list m

[issue23441] rlcompleter: tab on empty prefix => insert spaces

2015-07-09 Thread David Beazley
David Beazley added the comment: Wanted to add: I see this as being about the same as having a broken window pane on the front of Python 3. Maybe there are awesome things inside, but it makes a bad first impression on anyone who dares to use the interactive console

[issue23441] rlcompleter: tab on empty prefix => insert spaces

2015-07-09 Thread David Beazley
David Beazley added the comment: Frivolity aside, I really wish this issue would get more traction and a fix. Indentation is an important part of the Python language (obviously). A pretty standard way to indent is to hit "tab" in whatever environment you're using to edit Py

[issue23441] rlcompleter: tab on empty prefix => insert spaces

2015-07-07 Thread David Beazley
David Beazley added the comment: For what it's worth, I'm kind of tired having to hack site.py every time I upgrade Python in order to avoid being shown 6000 choices when hitting tab on an empty line. It is crazy annoying. -- ___ Pyth

[issue23441] rlcompleter: tab on empty prefix => insert spaces

2015-07-07 Thread David Beazley
David Beazley added the comment: This is a problem that will never be fixed. Sure, it was a release blocker in Python 3.4. It wasn't fixed. It is a release blocker in Python 3.5. It won't be fixed. They'll just tell you to indent using the spacebar as generations of typists

[issue23642] Interaction of ModuleSpec and C Extension Modules

2015-03-12 Thread David Beazley
David Beazley added the comment: This is great news. Read the PEP draft and think this is a very good thing to be addressing. Thanks, Brett. -- ___ Python tracker <http://bugs.python.org/issue23

[issue23642] Interaction of ModuleSpec and C Extension Modules

2015-03-12 Thread David Beazley
David Beazley added the comment: Note: Might be related to Issue 19713. -- ___ Python tracker <http://bugs.python.org/issue23642> ___ ___ Python-bugs-list mailin

[issue23642] Interaction of ModuleSpec and C Extension Modules

2015-03-11 Thread David Beazley
David Beazley added the comment: Sorry. I take back the previous message. It still doesn't quite do what I want. Anyways, any insight or thoughts about this would be appreciated ;-). -- ___ Python tracker <http://bugs.python.org/is

[issue23642] Interaction of ModuleSpec and C Extension Modules

2015-03-11 Thread David Beazley
David Beazley added the comment: inal comment. It seems that one can generally avoid a lot of nastiness if importlib.reload() is used instead. For example: >>> mod = sys.modules[spec.name] = module_from_spec(spec) >>> importlib.reload(mod) This works for both source and

[issue23642] Interaction of ModuleSpec and C Extension Modules

2015-03-11 Thread David Beazley
New submission from David Beazley: I have been investigating some of the new importlib machinery and the addition of ModuleSpec objects. I am a little curious about the intended handling of C Extension modules going forward. Backing up for a moment, consider a pure Python module. It seems

[issue15986] memoryview: expose 'buf' attribute

2014-07-31 Thread David Beazley
David Beazley added the comment: One of the other goals of memoryviews is to make memory access less hacky. To that end, it would be nice to have the .buf attribute available given that all of the other attributes are already there. I don't see why people should need to do some even

[issue15986] memoryview: expose 'buf' attribute

2014-07-31 Thread David Beazley
David Beazley added the comment: Well, a lot of things in this big bad world are dangerous. Don't see how this is any more dangerous than all of the peril that tools like ctypes and llvmpy already provide. -- ___ Python tracker

[issue15986] memoryview: expose 'buf' attribute

2014-07-31 Thread David Beazley
David Beazley added the comment: There are other kinds of libraries that might want to access the .buf attribute. For example, the llvmpy extension. Exposing it would be useful. -- ___ Python tracker <http://bugs.python.org/issue15

[issue5845] rlcompleter should be enabled automatically

2014-04-08 Thread David Beazley
David Beazley added the comment: Funny thing, this feature breaks the interactive interpreter in the most basic way on OS X systems. For example, the tab key won't even work to indent. You can't even type the most basic programs into the interactive interpreter. For example: &g

[issue18111] Add a default argument to min & max

2013-06-04 Thread David Beazley
David Beazley added the comment: To me, the fact that m = max(s) if s else default doesn't work with iterators alone makes this worthy of consideration. I would also note that min/max are the only reduction functions that don't have the ability to work with a possibly empty sequ

[issue18111] Add a default argument to min & max

2013-06-03 Thread David Beazley
David Beazley added the comment: I could have used this feature myself somewhat recently. It was in some code involving document matching where zero or more possible candidates were assigned a score and I was trying to find the max score. The fact that an empty list was a possibility

[issue16723] io.TextIOWrapper on urllib.request.urlopen terminates prematurely

2013-01-15 Thread David Beazley
David Beazley added the comment: I have run into this bug myself. Agree that a file-like object should never report itself as closed unless .close() has been explicitly called on it. HTTPResponse should not return itself as closed after the end-of-file has been reached. I think there is

[issue16894] Function attribute access doesn't invoke methods in dict subclasses

2013-01-08 Thread David Beazley
New submission from David Beazley: Suppose you subclass a dictionary: class mdict(dict): def __getitem__(self, index): print('Getting:', index) return super().__getitem__(index) Now, suppose you define a function and perform these steps that reassign the

[issue14965] super() and property inheritance behavior

2013-01-06 Thread David Beazley
David Beazley added the comment: Just as a note, there is a distinct possibility that a "property" in a superclass could be some other kind of descriptor object that's not a property. To handle that case, the solution of super(self.__class__, self.__class__).x.fset(self

[issue16254] PyUnicode_AsWideCharString() increases string size

2012-10-16 Thread David Beazley
David Beazley added the comment: Another note: the PyUnicode_AsUTF8String() doesn't leave the UTF-8 encoded byte string behind on the original string object. I got into this thinking that PyUnicode_AsWideCharString() might have similar beh

[issue16254] PyUnicode_AsWideCharString() increases string size

2012-10-16 Thread David Beazley
David Beazley added the comment: Maybe it's not a bug, but I still think it's undesirable. Basically, you have a function that allocates a buffer, fills it with data, and allows the buffer to be destroyed. Yet, as a side effect, it allocates a second buffer, fills it, and p

[issue16254] PyUnicode_AsWideCharString() increases string size

2012-10-16 Thread David Beazley
David Beazley added the comment: I should quickly add, is there any way to simply have this function not keep the wchar_t buffer around afterwards? That would be great. -- ___ Python tracker <http://bugs.python.org/issue16

[issue16254] PyUnicode_AsWideCharString() increases string size

2012-10-16 Thread David Beazley
New submission from David Beazley: The PyUnicode_AsWideCharString() function is described as creating a new buffer of type wchar_t allocated by PyMem_Alloc() (which must be freed by the user). However, if you use this function, it causes the size of the original string object to permanently

[issue16132] ctypes incorrectly encodes .format attribute of memory views

2012-10-04 Thread David Beazley
New submission from David Beazley: This is somewhat related to an earlier bug report concerning memory views, but as far as I can tell, ctypes is not encoding the '.format' attribute correctly in most cases. Consider this example: First, create a ctypes array: >>> a = (

[issue15944] memoryviews and ctypes

2012-09-20 Thread David Beazley
David Beazley added the comment: One followup note---I think it's fine to punt on cast('B') if the memoryview is non-contiguous. That's a rare case that's probably not as common. -- ___ Python tracker <http:

[issue15944] memoryviews and ctypes

2012-09-20 Thread David Beazley
David Beazley added the comment: There's probably a bigger discussion about memoryviews for a rainy day. However, the number one thing that would save all of this in my book would be to make sure cast('B') is universally supported regardless of format including endianness--es

[issue15944] memoryviews and ctypes

2012-09-14 Thread David Beazley
David Beazley added the comment: I should add that 0-dim indexing doesn't work as described either: >>> import ctypes >>> d = ctypes.c_double() >>> m = memoryview(d) >>> m[()] Traceback (most recent call last): File "", line 1, in

[issue15944] memoryviews and ctypes

2012-09-14 Thread David Beazley
David Beazley added the comment: Just to be specific, why is something like this not possible? >>> d = ctypes.c_double() >>> m = memoryview(d) >>> m[0:8] = b'abcdefgh' >>> d.value 8.540883223036124e+194 >>> (Doesn't have to be exac

[issue15944] memoryviews and ctypes

2012-09-14 Thread David Beazley
David Beazley added the comment: No, I want to be able to access the raw bytes sitting behind a memoryview as bytes without all of this casting and reinterpretation. Just show me the raw bytes. Not doubles, not ints, not structure packing, not copying into byte strings, or whatever. Is

[issue15944] memoryviews and ctypes

2012-09-14 Thread David Beazley
David Beazley added the comment: I don't think memoryviews should be imposing any casting restrictions at all. It's low level. Get out of the way. -- ___ Python tracker <http://bugs.python.o

[issue15944] memoryviews and ctypes

2012-09-14 Thread David Beazley
David Beazley added the comment: Even with the <http://bugs.python.org/issue15944> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%

[issue15944] memoryviews and ctypes

2012-09-14 Thread David Beazley
David Beazley added the comment: I don't want to read the representation by copying it into a bytes object. I want direct access to the underlying memory--including the ability to modify it. As it stands now, it's complete

[issue15944] memoryviews and ctypes

2012-09-14 Thread David Beazley
New submission from David Beazley: I've been playing with the interaction of ctypes and memoryviews and am curious about intended behavior. Consider the following: >>> import ctypes >>> d = ctypes.c_double() >>> m = memoryview(d) >>> m.ndim 0 >>&

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-03 Thread David Beazley
David Beazley added the comment: File attached.The file can be read in its entirety in binary mode. -- Added file: http://bugs.python.org/file26673/access-log-0108.bz2 ___ Python tracker <http://bugs.python.org/issue15

[issue15546] Iteration breaks with bz2.open(filename,'rt')

2012-08-03 Thread David Beazley
New submission from David Beazley: The bz2 library in Python3.3b1 doesn't support iteration for text-mode properly. Example: >>> f = bz2.open('access-log-0108.bz2') >>> next(f) # Works b'140.180.132.213 - - [24/Feb/2008:00:08:59 -0600] "GET /

[issue10791] Wrapping TextIOWrapper around gzip files

2011-02-23 Thread David Beazley
David Beazley added the comment: Python 3.2 (r32:88445, Feb 20 2011, 21:51:21) [GCC 4.2.1 (Apple Inc. build 5664)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import gzip >>> import io >&

[issue10791] Wrapping TextIOWrapper around gzip files

2011-02-23 Thread David Beazley
David Beazley added the comment: If I can find some time, I may took a look at this. I just noticed that similar problems arise trying to wrap TextIOWrapper around the file-like objects returned by urllib.request.urlopen as well. In the big picture, some discussion of what it means to be

[issue10791] Wrapping TextIOWrapper around gzip files

2011-02-23 Thread David Beazley
David Beazley added the comment: Bump. This is still broken in Python 3.2. -- ___ Python tracker <http://bugs.python.org/issue10791> ___ ___ Python-bugs-list m

[issue11117] Implementing Async IO

2011-02-05 Thread David Beazley
David Beazley added the comment: Glad you liked it! I think there is a bit of a cautionary tale in there though. With aio_, there is the promise of better performance, but you're also going to need a *LOT* of advance planning and thought to avoid creating a tangled coding nightmare wi

[issue11117] Implementing Async IO

2011-02-05 Thread David Beazley
David Beazley added the comment: Anyone contemplating the use of aio_ functions should first go read "The Story of Mel". http://www.catb.org/jargon/html/story-of-mel.html -- nosy: +dabeaz ___ Python tracker <http://bugs.python.o

[issue7322] Socket timeout can cause file-like readline() method to lose data

2011-01-14 Thread David Beazley
David Beazley added the comment: Just wanted to say that I agree it's nonsense to continue reading on a socket that timed out (I'm not even sure what I might have been thinking when I first submitted this bug other than just experimenting with edge cases of the socket interface

[issue10907] OS X installer: warn users of buggy Tcl/Tk in OS X 10.6

2011-01-14 Thread David Beazley
David Beazley added the comment: A comment from the training world: The instability of IDLE on the Mac makes teaching introductory Python courses a nightmare at the moment. Sure, one might argue that students should install an alternative editor, but then you usually end up with two

[issue7322] Socket timeout can cause file-like readline() method to lose data

2011-01-13 Thread David Beazley
David Beazley added the comment: Have any other programming environments ever had a feature where a socket timeout returns an exception containing partial data?I'm not aware of one offhand and speaking as a systems programmer, something like this might be somewhat unexpected. My co

[issue10791] Wrapping TextIOWrapper around gzip files

2010-12-29 Thread David Beazley
David Beazley added the comment: Hmmm. Interesting. In the big picture, it might be an interesting project for someone (not necessarily the core devs) to sit down and refactor both of these modules so that they play nice with Python 3 I/O system. Obviously that's a project outsid

[issue10791] Wrapping TextIOWrapper around gzip files

2010-12-29 Thread David Beazley
David Beazley added the comment: Do Python devs really view gzip and bz2 as two totally completely different animals? They both have the same functionality and would be used for the same kinds of things. Maybe I'm missing some

[issue10791] Wrapping TextIOWrapper around gzip files

2010-12-29 Thread David Beazley
David Beazley added the comment: C or not, wrapping a BZ2File instance with a TextIOWrapper to get text still seems like something that someone might want to do. I doubt it would take much modification to give BZ2File instances the required set of methods

[issue10791] Wrapping TextIOWrapper around gzip files

2010-12-29 Thread David Beazley
David Beazley added the comment: It goes without saying that this also needs to be checked with the bz2 module. A quick check seems to indicate that it has the same problem. While you're at it, maybe someone could add an 'open' function to bz2 to make it symmetrical wi

[issue10791] Wrapping TextIOWrapper around gzip files

2010-12-29 Thread David Beazley
New submission from David Beazley : Is something like this supposed to work: >>> import gzip >>> import io >>> f = io.TextIOWrapper(gzip.open("foo.gz"),encoding='ascii')) Traceback (most recent call last): File "", line 1, in Attri

[issue10783] struct.pack() and Unicode strings

2010-12-28 Thread David Beazley
David Beazley added the comment: Thanks everyone for looking at this! -- ___ Python tracker <http://bugs.python.org/issue10783> ___ ___ Python-bugs-list mailin

[issue10783] struct.pack() and Unicode strings

2010-12-28 Thread David Beazley
David Beazley added the comment: As a user of Python 3, I would like echo Victor's comment about fixing the API right now as opposed to having to deal with it later. I can only speak for myself, but I would guess that anyone using Python 3 already understands that it's bleedin

[issue10783] struct.pack() and Unicode strings

2010-12-27 Thread David Beazley
David Beazley added the comment: Actually, here's another one of my favorite examples: >>> import struct >>> struct.pack("s","\xf1") b'\xc3' >>> Not only does this not encode the correct value, it doesn't even encode the ent

[issue10783] struct.pack() and Unicode strings

2010-12-27 Thread David Beazley
David Beazley added the comment: I encountered this issue is in the context of distributed computing/interprocess communication involving binary-encoded records (and encoding/decoding such records using struct). At its core, this is all about I/O--something where encodings and decoding

[issue10783] struct.pack() and Unicode strings

2010-12-27 Thread David Beazley
David Beazley added the comment: Why is it even encoding at all? Almost every other part of Python 3 forces you to be explicit about bytes/string conversion. For example: struct.pack("10s", x.encode('utf-8')) Given that automatic conversion is documented, it's n

[issue10783] struct.pack() and Unicode strings

2010-12-27 Thread David Beazley
David Beazley added the comment: Hmmm. Well, the docs seem to say that it's allowed and that it will be encoded as UTF-8. Given the treatment of Unicode/bytes elsewhere in Python 3, all I can say is that this behavior is rather surpr

[issue10783] struct.pack() and Unicode strings

2010-12-27 Thread David Beazley
David Beazley added the comment: Note: This is what happens in Python 2.6.4: >>> import struct >>> struct.pack("10s",u"Jalape\u00f1o") Traceback (most recent call last): File "", line 1, in

[issue10783] struct.pack() and Unicode strings

2010-12-27 Thread David Beazley
New submission from David Beazley : Is the struct.pack() function supposed to automatically encode Unicode strings into binary? For example: >>> struct.pack("10s","Jalape\u00f1o") b'Jalape\xc3\xb1o\x00' >>> This is Python 3.2b1. --

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-28 Thread David Beazley
David Beazley added the comment: Wow, that is a *really* intriguing performance result with radically different behavior than Unix. Do you have any ideas of what might be causing it? -- ___ Python tracker <http://bugs.python.org/issue7

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-27 Thread David Beazley
David Beazley added the comment: One more attempt at fixing tricky segfaults. Glad someone had some eagle eyes on this :-). -- Added file: http://bugs.python.org/file17106/dabeaz_gil.patch ___ Python tracker <http://bugs.python.org/issue7

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-27 Thread David Beazley
Changes by David Beazley : Removed file: http://bugs.python.org/file17104/dabeaz_gil.patch ___ Python tracker <http://bugs.python.org/issue7946> ___ ___ Python-bugs-list m

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-27 Thread David Beazley
David Beazley added the comment: I stand corrected. However, I'm going to have to think of a completely different approach for carrying out that functionality as I don't know how the take_gil() function is able to determine whether gil_last_holder has been deleted or not.

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-27 Thread David Beazley
David Beazley added the comment: That second access of gil_last_holder->cpu_bound is safe because that block of code is never entered unless some other thread currently holds the GIL. If a thread holds the GIL, then gil_last_holder is guaranteed to have a valid va

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-27 Thread David Beazley
David Beazley added the comment: Added extra pointer check to avoid possible segfault. -- Added file: http://bugs.python.org/file17104/dabeaz_gil.patch ___ Python tracker <http://bugs.python.org/issue7

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-27 Thread David Beazley
Changes by David Beazley : Removed file: http://bugs.python.org/file17102/dabeaz_gil.patch ___ Python tracker <http://bugs.python.org/issue7946> ___ ___ Python-bugs-list m

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-26 Thread David Beazley
David Beazley added the comment: New version of patch that will probably fix Windows-XP problems. Was doing something stupid in the monitor (not sure how it worked on Unix). -- Added file: http://bugs.python.org/file17102/dabeaz_gil.patch

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-26 Thread David Beazley
Changes by David Beazley : Removed file: http://bugs.python.org/file17094/dabeaz_gil.patch ___ Python tracker <http://bugs.python.org/issue7946> ___ ___ Python-bugs-list m

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-26 Thread David Beazley
David Beazley added the comment: I've also attached a new file schedtest.py that illustrates a subtle difference between having the GIL monitor thread and not having the monitor. Without the monitor, every thread is responsible for its own scheduling. If you have a lot of threads ru

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-26 Thread David Beazley
David Beazley added the comment: I've updated the GIL patch to reflect concerns about the monitor thread running forever. This version has a suspension mechanism where the monitor goes to sleep if nothing is going on for awhile. It gets resumed if threads try to acquire the GIL

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-26 Thread David Beazley
Changes by David Beazley : Removed file: http://bugs.python.org/file17084/dabeaz_gil.patch ___ Python tracker <http://bugs.python.org/issue7946> ___ ___ Python-bugs-list m

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-26 Thread David Beazley
David Beazley added the comment: Greg, I like the idea of the monitor suspending if no thread owns the GIL. Let me work on that. Good point on embedded systems. Antoine, Yes, the gil monitor is completely independent and simply ticks along every 5 ms. A worst case scenario is that an

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-25 Thread David Beazley
David Beazley added the comment: Here is the result of running the writes.py test with the patch I submitted. This is on OS-X. bash-3.2$ ./python.exe writes.py t1 2.83990693092 0 t2 3.27937912941 0 t1 5.54346394539 1 t2 6.68237304688 1 t1 8.9648039341 2 t2 9.60041999817 2 t1 12.1856160164 3

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-25 Thread David Beazley
David Beazley added the comment: One comment on that patch I just submitted. Basically, it's an attempt to make an extremely simple tweak to the GIL that fixes most of the problems discussed here in an extremely simple manner. I don't have any special religious attachment to

[issue7946] Convoy effect with I/O bound threads and New GIL

2010-04-25 Thread David Beazley
David Beazley added the comment: The attached patch makes two simple refinements to the new GIL implemented in Python 3.2. Each is briefly described below. 1. Changed mechanism for thread time expiration In the current implementation, threads perform a timed-wait on a condition variable

[issue8532] Refinements to Python 3 New GIL

2010-04-25 Thread David Beazley
David Beazley added the comment: Can't decide whether this should be attached to Issue 7946 or not. I will also post it there. (Feel free to close this issue if you want to keep 7946 alive). -- ___ Python tracker <http://bugs.python.org/i

[issue8532] Refinements to Python 3 New GIL

2010-04-25 Thread David Beazley
New submission from David Beazley : The attached patch makes two simple refinements to the new GIL implemented in Python 3.2. Each is briefly described below. 1. Changed mechanism for thread time expiration In the current implementation, threads perform a timed-wait on a condition variable

[issue8410] Fix emulated lock to be 'fair'

2010-04-21 Thread David Beazley
David Beazley added the comment: I know that multicore processors are all the rage right now, but one thing that concerns me about this patch is its effect on single-core systems. If you apply this on a single-CPU, are threads just going to sit there and thrash as they rapidly context

[issue8299] Improve GIL in 2.7

2010-04-18 Thread David Beazley
David Beazley added the comment: Here are the results of running the fair.py test on a Mac OS-X system using a "fair" GIL implementation (modified condition variable): [ Fair GIL, Dual-Core, OS-X ] Sequential execution slow: 5.490943 (0 left) fast: 0.369257 (0 left) Threaded exec

[issue8299] Improve GIL in 2.7

2010-04-17 Thread David Beazley
David Beazley added the comment: As a followup, since I'm not sure anyone actually here actually tried a fair GIL on Linux, I incorporated your suggested fairness patch to the condition-variable version of the GIL (using this pseudocode you wrote as a guide): with gil.cond:

[issue8299] Improve GIL in 2.7

2010-04-17 Thread David Beazley
David Beazley added the comment: I'm definitely sure that semaphores were being used in my test---I stuck a print statement inside the code that creates locks just to make sure it was using the semaphore version :-). Unfortunately, at this point I think most of this discussion is aca

[issue8299] Improve GIL in 2.7

2010-04-16 Thread David Beazley
David Beazley added the comment: One other comment. Running the modified fair.py file on my Linux system using Python compiled with semaphores shows they they are *definitely* not fair. Here's the relevant part of your test: Treaded, balanced execution, with quickstop: fast C: 1.5808

  1   2   >