[issue37270] Manage memory lifetime for all type-related objects.
Change by Joe Jevnik : -- keywords: +patch pull_requests: +13925 stage: -> patch review pull_request: https://github.com/python/cpython/pull/14066 ___ Python tracker <https://bugs.python.org/issue37270> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37270] Manage memory lifetime for all type-related objects.
New submission from Joe Jevnik : When using PyType_FromSpec, the memory for PyType_Spec.name, Py_tp_methods, and Py_tp_members needs to somehow outlive the resulting type. This makes it hard to use this interface to generate types without just leaking the memory for these arrays, which is bad if you are programatically creating short-lived types. I posted about this on capi-sig, and the response seemed to be that it would be to replace the things that currently hold pointers into the array with copies of the data. Remove internal usages of PyMethodDef and PyGetSetDef. For PyMethodDef, change PyCFunctionObject to replace the PyMethodDef* member with a PyCFunctionBase member. The PyCFunctionBase is a new struct to hold the managed values of a PyMethodDef. This type is shared between PyCFunction and the various callable descriptor objects. A PyCFunctionBase is like a PyMethodDef but replaces the char* members with PyObject* members. For PyGetSetDef, inline the members on the resulting PyGetSetDescrObject, replacing all char* members with PyObject* members. The memory for the closure is *not* managed, adding support for that would likely require an API change and can be done in a future change. For the tp_name field, instead of setting it directly to the value of PyType_Spec.name, set it to the result of PyUnicode_AsUTF8(ht_name), where ht_name is the PyUnicode object created from the original spec name. This is the same trick used to properly manage this pointer for heap types when the __name__ is reassigned. -- components: Extension Modules messages: 345539 nosy: ll priority: normal severity: normal status: open title: Manage memory lifetime for all type-related objects. versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue37270> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36068] Make _tuplegetter objects serializable
Joe Jevnik added the comment: Thank you for reviewing this so quickly! -- ___ Python tracker <https://bugs.python.org/issue36068> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36068] Make _tuplegetter objects serializable
Change by Joe Jevnik : -- keywords: +patch pull_requests: +12006 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue36068> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36068] Make _tuplegetter objects serializable
New submission from Joe Jevnik : The new _tuplegetter objects for accessing fields of a namedtuple are no longer serializable with pickle. Cloudpickle, a library which provides extensions to pickle to facilitate distributed computing in Python, depended on being able to pickle the members of namedtuple classes. While property isn't serializable, cloudpickle has support for properties allowing us to serialize the old property(itemgetter) members. The attached PR adds a __reduce__ method to _tuplegetter objects which will allow serialization without special support. Another option would be to expose `index` as a read-only attribute, allowing cloudpickle or other libraries to provide the pickle implementation as a third-party library. -- components: Library (Lib) messages: 336251 nosy: ll priority: normal severity: normal status: open title: Make _tuplegetter objects serializable type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue36068> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23927] getargs.c skipitem() doesn't skip 'w*'
Change by Joe Jevnik : -- pull_requests: +7946 ___ Python tracker <https://bugs.python.org/issue23927> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
Change by Joe Jevnik : -- pull_requests: +7754 ___ Python tracker <https://bugs.python.org/issue23926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28638] Optimize namedtuple creation
Joe Jevnik added the comment: I added a benchmark suite (using Victor's perf utility) to cnamedtuple. The results are here: https://github.com/ll/cnamedtuple#benchmarks To summarize: type creation is much faster; instance creation and named attribute access are a bit faster. -- nosy: +ll ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28638> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13349] Non-informative error message in index() and remove() functions
Changes by Joe Jevnik <j...@quantopian.com>: -- pull_requests: +1919 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue13349> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30477] tuple.index error message improvement
Joe Jevnik added the comment: As a more meta question: I have noticed that many error messages in the stdlib use PyErr_SetString, or choose to use the type name instead of the repr of the object. Is there a reason for this? I normally try to fill the error message with as much context as possible to make debugging easier. Is it a performance worry? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30477> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30477] tuple.index error message improvement
Joe Jevnik added the comment: I agree, "%R in tuple" is enough information for me. This would also remove the need to manually repr the object. -- type: enhancement -> ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30477> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30477] tuple.index error message improvement
New submission from Joe Jevnik: The old error of tuple.index(x): x not in tuple seemed very confusing to me. It was also harder to quickly understand what the program was doing wrong. This saves people a second pass through the program under the debugger because they can just see what the invalid value was. The reason I am explicitly calling repr instead of using %R is that I didn't want to call repr twice if I didn't need to. If people think that is fine the format can just be tuple.index(%R): %R not in tuple instead. -- messages: 294516 nosy: brett.cannon, ll priority: normal pull_requests: 1906 severity: normal status: open title: tuple.index error message improvement versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30477> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30180] PyArg_ParseTupleAndKeywords supports required keyword only arguments
New submission from Joe Jevnik: I opened a pr to remove a line in the docs about $ needing to follow | in PyArg_ParseTupleAndKeywords. In practice, you can just use a $ to create required keyword arguments which intuitively makes sense. I was told this should raise a SystemError; however, you can create required keyword only arguments in Python so I am not sure why we would want to fail when this is done with PyArg_ParseTupleAndKeywords. -- messages: 292385 nosy: ll, serhiy.storchaka priority: normal pull_requests: 1417 severity: normal status: open title: PyArg_ParseTupleAndKeywords supports required keyword only arguments ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30180> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30180] PyArg_ParseTupleAndKeywords supports required keyword only arguments
Changes by Joe Jevnik <j...@quantopian.com>: -- assignee: -> docs@python components: +Documentation nosy: +docs@python ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30180> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29468] zipfile should catch ValueError as well as OSError to detect bad seek calls
New submission from Joe Jevnik: In zipfile.py only OSError is checked to see if seek fails: ``` def _EndRecData64(fpin, offset, endrec): """ Read the ZIP64 end-of-archive records and use that to update endrec """ try: fpin.seek(offset - sizeEndCentDir64Locator, 2) except OSError: # If the seek fails, the file is not large enough to contain a ZIP64 # end-of-archive record, so just return the end record we were given. return endrec ``` I belive that this should also catch ValueError so that other file-like objects may be passed to ZipFile. The particular case I ran into was passing an mmap object: ``` """ $ python p.py sys.version_info(major=3, minor=6, micro=0, releaselevel='final', serial=0) [] Traceback (most recent call last): File "p.py", line 34, in with mmap_shared_raw_zipfile(f.name) as zf: File "/usr/lib64/python3.6/contextlib.py", line 82, in __enter__ return next(self.gen) File "p.py", line 23, in mmap_shared_raw_zipfile ZipFile(mm) as zf: File "/usr/lib64/python3.6/zipfile.py", line 1100, in __init__ self._RealGetContents() File "/usr/lib64/python3.6/zipfile.py", line 1163, in _RealGetContents endrec = _EndRecData(fp) File "/usr/lib64/python3.6/zipfile.py", line 264, in _EndRecData return _EndRecData64(fpin, -sizeEndCentDir, endrec) File "/usr/lib64/python3.6/zipfile.py", line 196, in _EndRecData64 fpin.seek(offset - sizeEndCentDir64Locator, 2) ValueError: seek out of range """ from contextlib import contextmanager import mmap import sys from tempfile import NamedTemporaryFile from zipfile import ZipFile print(sys.version_info) @contextmanager def mmap_shared_raw_zipfile(path): """Open a zipfile with mmap shared so the data can be shared in multiple processes. Parameters -- path : str The path the zipfile to open. Notes - The context manager returns a :class:`zipfile.ZipFile` on enter. """ with open(path) as f, \ mmap.mmap(f.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ) as mm, \ ZipFile(mm) as zf: yield zf with NamedTemporaryFile() as f: ZipFile(f, mode='w').close() print(ZipFile(f.name).infolist()) with NamedTemporaryFile() as f: ZipFile(f, mode='w').close() with mmap_shared_raw_zipfile(f.name) as zf: print(zf.infolist()) ``` -- components: Library (Lib) messages: 287185 nosy: ll priority: normal severity: normal status: open title: zipfile should catch ValueError as well as OSError to detect bad seek calls versions: Python 3.6 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29468> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29096] Signal Handlers reliably cause UnboundLocalErrors
Joe Jevnik added the comment: The issue appears to be in ceval.c:unicode_concatenate (or the py2 equivalent) The code sets the local variable on the lhs to NULL before doing a potentially inplace append to the string. This means that if a signal is raised during the concat, we never hit the STORE_FAST instruction following the INPLACE_ADD so the local or cell still holds NULL, triggering an error. I am not really sure what can be done to prevent the failure while still allowing the optimization. -- components: +Interpreter Core nosy: +ll type: crash -> ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29096> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled
Joe Jevnik added the comment: I definitely could have used PyImport_Import and PyObject_Call to accomplish this task; however, when I looked at at the implementation of these functions it seemed like a lot of overhead and book keeping just to set a boolean. Since I was already in a C extension it would be nicer to not need to worry about PyObjects and ref counts just to set this value so I thought it would be nice to expose these small functions to users. Since this is equivalent to gc.enable or gc.disable I don't think there is a deep can of worms here. The only difference in the implementation is that it doesn't return None. There is already PyGC_Collect, so I figured these were just omitted because no one thought to add them. I don't have a burning desire to get this in, I just thought it would be a nice to have. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28254> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled
New submission from Joe Jevnik: I was writing an extension module that was working with weakrefs and wanted to ensure that the GC would not run for a block of code. I noticed that gc.enable/gc.disable are not exposed to C and the state of the gc is in a static variable so it cannot be mutated from an extension. This change adds an easier way to access this functionality in the C api without going through an import or PyObject_Call. I am wondering where to document these functions as well as PyGC_Collect. I didn't see that function anywhere in the docs (and didn't know about it) so I would like to make them more visible. My first thought was Doc/c-api/gcsupport.rst but this is not in service of supporting the GC, it is about the state of the GC itself. If that seems like a decent place I could add a small section at the bottom about interacting with the GC and document these new functions and PyGC_Collect. I'm sorry if this is exposed elsewhere. If it is I can try to add some links to i -- components: Extension Modules files: gc-capi.patch keywords: patch messages: 277245 nosy: ll priority: normal severity: normal status: open title: Add C API for gc.enable, gc.disable, and gc.isenabled versions: Python 3.7 Added file: http://bugs.python.org/file44787/gc-capi.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28254> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27241] Catch exceptions raised in pstats add (repl)
Joe Jevnik added the comment: bump -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27241> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27359] OrderedDict pseudo-literals (WIP)
Joe Jevnik added the comment: Sorry about undoing the settings, that was unintentional. The UI around these settings is sort of unintuitive. I was waiting to propose this on python-dev until I got it working, but I wanted to push up what I did have as a work in progress so I could try to get some help with the crash I am seeing. In the future I will upload the patch somewhere else until it is done. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27359> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27359] OrderedDict pseudo-literals (WIP)
Joe Jevnik added the comment: Assuming all dicts are ordered, there will be no need for this (as long as the parser preserves order) so I am okay with dropping this. One of the main use cases I have for wanting nice literals is when I am working on tests. I find it very nice to be able to write out the expected results in a clear way. While the app code might not be using the literals, it improves the readability of the tests, which make it easier to see that the test is correct. -- resolution: later -> status: closed -> open ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27359> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27359] OrderedDict pseudo-literals (WIP)
New submission from Joe Jevnik: This proposes the following syntax for creating OrderedDict literals: OrderedDict[k1: v1, k2: v2, ...] This is implemented by putting a metaclass on OrderedDict which has a getitem that turns the slices into a list of tuples (after validation). The idea is taken from the library datashape where we have done this to implement pseudo literals for the Record type. This is marked as WIP because I am not quite sure why, but this patch is causing subclasses of OrderedDict to segfault in PyObject_SetItem. -- components: Library (Lib) files: odict_literals.patch keywords: patch messages: 268954 nosy: ll priority: normal severity: normal status: open title: OrderedDict pseudo-literals (WIP) versions: Python 3.6 Added file: http://bugs.python.org/file43495/odict_literals.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27359> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27241] Catch exceptions raised in pstats add (repl)
New submission from Joe Jevnik: I was trying to add a file and accidently mistyped the name which crashed the repl session. I think it would be nicer to report the failure but bring me back to the prompt so I can decide what I would like to do. This patch catches any IOErrors that can be raised when adding a file in the repl, reports it to the user, and then returns them to the repl. -- components: Demos and Tools files: catch-pstats-add.patch keywords: patch messages: 267542 nosy: ll priority: normal severity: normal status: open title: Catch exceptions raised in pstats add (repl) type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file43262/catch-pstats-add.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27241> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27039] bytearray.remove cannot remove bytes with value greater than 127
Joe Jevnik added the comment: This seems to just be a bug in the implementation of remove. I have a patch to fix this and a test case. -- keywords: +patch nosy: +ll Added file: http://bugs.python.org/file42875/bytearray-remove.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27039> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: people have had some time to think about this, whats should the name be so we can merge this? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26802] Avoid copy in call_function_var when no extra stack args are passed
Joe Jevnik added the comment: CALL_FUNCTION doesn't use extended arg; I don't see what we get by adding some opcode to convert the sequence into a tuple. We also don't want to box up the stack arguments into a tuple because when we do a CALL_FUNCTION to a python function we just copy the stack[-n:] elements into the locals of the next frame. Emitting a build_tuple here would be an unneeded allocation. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26802> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26802] Avoid copy in call_function_var when no extra stack args are passed
Joe Jevnik added the comment:
> It's possible that stararg is already an empty tuple. Is it worth to use it?
PyTuple_New(0) should return the unit tuple in most cases:
#if PyTuple_MAXSAVESIZE > 0
if (size == 0 && free_list[0]) {
op = free_list[0];
Py_INCREF(op);
#ifdef COUNT_ALLOCS
tuple_zero_allocs++;
#endif
return (PyObject *) op;
}
Looking at this again I think that checking if stararg is nonnull is more
clear, I will update the patch (as call-function-var-3.patch). I cannot exactly
rewrite the if to use the control flow you show because that would cause
non-tuple subclasses to forward a stararg of () instead of copying it into a
normal tuple when no arguments are passed ont the stack.
--
Added file: http://bugs.python.org/file42521/call-function-var-3.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26802>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26802] Avoid copy in call_function_var when no extra stack args are passed
Changes by Joe Jevnik <j...@quantopian.com>: -- type: -> performance ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26802> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26802] Avoid copy in call_function_var when no extra stack args are passed
Joe Jevnik added the comment:
in _PyEval_EvalCodeWithName we will have already pulled the underlying array
out of the tuple subclass and the then we will reconstruct the vararg value in
the locals from this array. This means that with this patch we can get:
>>> class C(tuple): pass
...
>>> def f(*args): print(type(args))
...
>>> f(*C([1, 2, 3]))
The only case where we get potentially strange behavior is if we have a type
whose tp_call did not forward to PyArg_Parse* or the PyTupleObject api and went
though the abstract object interface. I ran the tests and the benchmark suite
and did not run into any issues here but I can see how this would be
potentially problematic behavior.
I have updated the code to also do a PyTuple_CheckExact against `stararg`. The
tests I ran are:
timeit("h(*(a, b, c, d, e, f, g))", 'def h(a, b, c, d, e, f, g): pass\na, b, c,
d, e, f, g = range(7)', number=int(10e7))
default: 17.191688070015516
patch: 14.060781285981648
There is also a really great case where you star unpack a tuple of literals
which gets pushed into the co_consts:
timeit("f(*('a', 'b', 'c', 'd', 'e', 'f', 'g'))", 'def f(a, b, c, d, e, f, g):
pass', number=int(10e7))
default: 13.842308042978402
patch: 9.696972723002546
This case seems far less common, but maybe someone has something like:
a = 1, 2, 3
...
f(*a)
--
type: performance ->
Added file: http://bugs.python.org/file42515/call-function-var.patch
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26802>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
Joe Jevnik added the comment: bump -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26802] Avoid copy in call_function_var when no extra stack args are passed
New submission from Joe Jevnik: When star unpacking positions into a function we can avoid a copy iff there are no extra arguments coming from the stack. Syntactically this looks like: `f(*args)` This reduces the overhead of the call by about 25% (~30ns on my machine) based on some light profiling of just * unpacking. I can imagine this having a slight impact on code that uses this feature in a loop. The cost is minimal when we need to make the copy because the int != 0 check is dwarfed by the allocation. I did not notice a significant change between the slow path and the original code. The macro benchmark suite did not show a difference on my machine but this is not much more complex code. -- components: Interpreter Core files: call-function-var.patch keywords: patch messages: 263702 nosy: ll priority: normal severity: normal status: open title: Avoid copy in call_function_var when no extra stack args are passed versions: Python 3.6 Added file: http://bugs.python.org/file42512/call-function-var.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26802> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26448] dis.findlabels ignores EXTENDED_ARG
Changes by Joe Jevnik <j...@quantopian.com>: -- nosy: +ll ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26448> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26379] zlib decompress as_bytearray flag
Joe Jevnik added the comment: The recipe you showed looks like it is very complicated for the expected use case of decompressing all of the data into a mutable buffer. One difference I see with this and struct.pack_into or socket.recv_into is that in both of those cases we know how large our buffer needs to be. With zlib.decompress there is not a simple way for users to preallocate a buffer to hold the result and need to resort to some looping with a lot of resizes. I think that returning either a mutable or immutable buffer hits 95% of use cases. If we go with the decompress_into method I would strongly advise putting decompress_as_bytearray into the standard library so that users do not need write this themselves. Thank you for pointing me at the bz2 and LZMA modules. When we come to an agreement on the API I will update bz2 and LZMA so that they match. Should I also update the gzip module? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: ping: any decision on this? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26379] zlib decompress as_bytearray flag
New submission from Joe Jevnik: Adds a keyword only flag to zlib.decompress and zlib.Decompress.decompress which marks that the return type should be a bytearray instead of bytes. The use case for this is reading compressed data into a mutable array to do further operations without requiring that the user copy the data first. Often, if a user is choosing to compress the data there might be a real cost to copying the uncompressed bytes into a mutable buffer. The impetus for this change comes from a patch for Pandas (https://github.com/pydata/pandas/pull/12359). I have also worked on a similar feature for blosc, another popular compression library for python (https://github.com/Blosc/python-blosc/pull/107). My hope is to fix the hacky solution presented in the pandas patch by supporting this feature directly in the compression libraries. As a side note: this is my first time using the argument clinic and I love it. It was so simple to add this argument, thank you very much for developing such an amazing tool! -- components: Extension Modules files: zlib.patch keywords: patch messages: 260424 nosy: ll priority: normal severity: normal status: open title: zlib decompress as_bytearray flag type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file41944/zlib.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: Is there a decision on the name? I can update the patch if needed. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: > and the latter might give the impression it was some sort of special > method/attribute when it was not. Wouldn't adding this be special because it is specifically reserved by CPython as an implementation detail? Also, would adding this name (__stub or __stub__) be sufficient to get this patch merged again. I would not mind investigating the gc_head issue after but I view that as a separate issue which is only exercised by this patch. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue24379> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: partial's unique usage is why I feel like it would be so jarring for them do be named differently. I would be happiest having this feature at all so I will change the name to 'kwargs' if you would like. I just want to be sure that my reasons for choosing this name in the first place ere understood. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: Thanks for pointing me at the refleak, uploading an update -- Added file: http://bugs.python.org/file41203/methodcaller-attrs-1.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: Added a test case for the mutation of keywords. -- Added file: http://bugs.python.org/file41204/methodcaller-attrs-2.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: I only want this feature to keep the usage close to functools.partial. I was actually sort of surprised to see that mutation of the dict held in partial, but I would rather be consistent. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
Joe Jevnik added the comment: Thanks for review, looking into the reference count issue. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25770] expose name, args, and kwargs from methodcaller
New submission from Joe Jevnik: This patch adds 3 properties to methodcaller objects for inspecting the object at runtime: 1. 'name': the name of the method to call 2. 'args': the position arguments to pass to the method 3. 'keywords': the keyword arguments to pass to the method args and keywords act like functools.partial (that is why I did not name it kwargs). I noticed that recently the repr changed to expose this information which helps in the debugging use case; however, this allows us to use the methodcaller to maybe construct a new methodcaller with different args, or call a different method with the same args. -- components: Library (Lib) files: methodcaller-attrs.patch keywords: patch messages: 255636 nosy: ll priority: normal severity: normal status: open title: expose name, args, and kwargs from methodcaller type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file41198/methodcaller-attrs.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25770> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
Joe Jevnik added the comment: bumping this issue -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23926> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: Sorry about the ideas thread. Thank you for merging this! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: I posted this to python Ideas on June 10th of this year. That is where we decided to rename it from `slice.literal` to `operator.subscript`. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
Joe Jevnik added the comment: bumping so that we don't forget about this. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23926 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: Any more comments? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
Joe Jevnik added the comment: Sorry it took so long to get back to this. I didn't realize this was still open. I have provided the update to the docs and moved it to the 3.6 section. I also made sure the patch still applies and the tests all pass. -- Added file: http://bugs.python.org/file39931/skipitems.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23926 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: I updated the docstring -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: updating to address the docs and order of assertions -- Added file: http://bugs.python.org/file39904/operator_subscript_pyonly.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: is it normal to get a lot of 500s when using the review system? -- Added file: http://bugs.python.org/file39906/operator_subscript_pyonly.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Changes by Joe Jevnik j...@quantopian.com: Added file: http://bugs.python.org/file39907/operator_subscript_pyonly.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Changes by Joe Jevnik j...@quantopian.com: Added file: http://bugs.python.org/file39913/operator_subscript_pyonly.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: Ah, I hadn't seen that, I will address those now, sorry about that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: updating with the slots change This also adds a ton of test cases -- Added file: http://bugs.python.org/file39902/operator_subscript_pyonly.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: ping: is there anything blocking this? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: I removed the C implementation. -- Added file: http://bugs.python.org/file39792/operator_subscript_pyonly.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: Based on some discussion on python-ideas, this is being renamed to operator.subscript. Here is the patch that makes the correct the changes to move this object into the operator module. -- components: +Extension Modules -Interpreter Core title: slice.literal notation - operator.subscript Added file: http://bugs.python.org/file39775/operator_subscript.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] operator.subscript
Joe Jevnik added the comment: I just moved it over since I implemented it for slice originally, I can drop the C implementation. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] slice.literal notation
Joe Jevnik added the comment: It is a singleton, does not accept the `maketuple` flag, and is written in C. I did not know about the s_ attribute of numpy before writing this; however, I still think that moving this object to slice improves code clarity (s_ is not a super clear name). I also think that this behaviour belongs on the slice object. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] slice.literal notation
Joe Jevnik added the comment: slice.literal[0] 0 y = slice.literal[1:2] slice(1, 2, None) slice.literal[0:1, ..., 3] (slice(0, 1, None), Ellipsis, 3) The way this object works right now does not create instances of some inner class of slice, instead, indexing it returns the key without modification. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24379] slice.literal notation
Joe Jevnik added the comment: Why not index the slice type itself? slice[1:2] I originally considered this and I personally really like this syntax, but I was concerned with ambiguity with the typing module The only question in my mind is what slice should do when given just a single index I think some of the power of this concept comes from the fact that I can express a complicated indexer without worrying about how it desugars. I would personally prefer being able to have this return tuples and scalars so that the syntax is easier to explain. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24379 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] property_descr_get reuse argument tuple
Joe Jevnik added the comment: I don't think that we need to worry about reusing the single argument tuple in a recursive situation because we never need the value after we start the call. We also just write our new value and then clean up with a NULL to make sure that we don't blow up when we dealloc the tuple. For example: class C(object): ... @property ... def f(self): ... return D().f ... class D(object): ... @property ... def f(self): ... return 1 ... C().f 1 This works with recursive properties. I also think that is is getting cleaned up on shutdown, if I put a pointer to garbage in the tuple, the interpreter blows up on shutdown; this makes me think that tuple_dealloc is being called somewhere. About putting the tuple on the property instance, that would nice for memory management; however, that increases the memory overhead of each property. This also means that we would only get the faster lookup after the property has been accessed once; this is fine but the current implementation makes it so that all properties are faster after any of them are looked up once. I could be wrong about the cleanup though. I am also updating the title and headers because this issue is no longer about namedtuple. -- components: +Interpreter Core -Extension Modules title: C implementation of namedtuple (WIP) - property_descr_get reuse argument tuple ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23910 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment: I switched to the static tuple. -- Added file: http://bugs.python.org/file39216/with-static-tuple.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23910 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment: I don't think that I can cache the __call__ of the fget object because it might be an instance of a heaptype, and if someone changed the __class__ of the object in between calls to another heaptype that had a different __call__, you would still get the __call__ from the first type. I also don't know if this is supported behavior or just something that works by accident. I read through PyObject_Call, and all the code is needed assuming we are not caching the tp_call value. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23910 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment:
I am currently on a different machine so these numbers are not relative to the
others posted earlier.
* default
./python -m timeit -s from collections import namedtuple as n;a = n('n', 'a b
c')(1, 2, 3) a.a
1000 loops, best of 3: 0.0699 usec per loop
* patch
./python -m timeit -s from collections import namedtuple as n;a = n('n', 'a b
c')(1, 2, 3) a.a
1000 loops, best of 3: 0.0468 usec per loop
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment:
I was unable to see a performance increase by playing with the
itemgetter.__call__ code; however, updating the propery code seemed to show a
small improvement. I think that for simple indexing the cost of checking if it
is a sequence outways the faster dispatch (when using PySequence_GetItem). I
can play with this further.
* default
[joejev@Sheila cpython]$ ./python -m timeit -s from collections import
namedtuple as n;a = n('n', 'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.101 usec per loop
* patch
[joejev@Sheila cpython]$ ./python -m timeit -s from collections import
namedtuple as n;a = n('n', 'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.0942 usec per loop
--
Added file: http://bugs.python.org/file39210/property.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment: This was very exciting, I have never run gprof before; so just to make sure I did this correctly, I will list my steps: 1. compile with the -pg flag set 1. run the test with ./python -m timeit ... 1. $ gprof python gmon.out profile.out Here is default: Each sample counts as 0.01 seconds. % cumulative self self total time seconds secondscalls Ts/call Ts/call name 22.48 0.87 0.87 PyEval_EvalFrameEx 9.82 1.25 0.38 PyObject_CallFunctionObjArgs 6.85 1.52 0.27 PyObject_GenericGetAttr 6.46 1.77 0.25 tupledealloc 5.56 1.98 0.22 PyArg_UnpackTuple 5.17 2.18 0.20 PyNumber_AsSsize_t 5.17 2.38 0.20 tuplesubscript 5.04 2.58 0.20 PyObject_Call 4.91 2.77 0.19 _PyType_Lookup 4.65 2.95 0.18 PyTuple_New 4.65 3.13 0.18 PyObject_GetItem 4.39 3.30 0.17 itemgetter_call 1.94 3.37 0.08 PyObject_GetAttr 1.81 3.44 0.07 PyObject_GC_UnTrack 1.81 3.51 0.07 _PyTuple_DebugMallocStats 1.03 3.55 0.04 PyErr_Occurred 1.03 3.59 0.04 property_descr_set 1.03 3.63 0.04 tuplerepr 0.78 3.66 0.03 PyLong_AsSsize_t 0.78 3.69 0.03 PyObject_GC_Track 0.52 3.71 0.02 property_descr_get 0.52 3.73 0.02 repeat_next 0.52 3.75 0.02 repeat_traverse 0.39 3.77 0.02 PyArg_ValidateKeywordArguments 0.39 3.78 0.02 _Py_CheckFunctionResult 0.26 3.79 0.01 PyCFunction_NewEx 0.26 3.80 0.01 PyCode_New 0.26 3.81 0.01 PyErr_Restore 0.26 3.82 0.01 PyType_GetSlot 0.26 3.83 0.01 _PyObject_CallMethodIdObjArgs 0.26 3.84 0.01 attrgetter_new 0.26 3.85 0.01 update_one_slot 0.13 3.86 0.01 _PyObject_GenericGetAttrWithDict 0.13 3.86 0.01 _PyObject_SetAttrId 0.13 3.87 0.01 gc_isenabled 0.13 3.87 0.01 visit_decref Here is my patch: Each sample counts as 0.01 seconds. % cumulative self self total time seconds secondscalls Ts/call Ts/call name 26.63 1.02 1.02 PyEval_EvalFrameEx 8.88 1.36 0.34 PyObject_GenericGetAttr 7.83 1.66 0.30 tupledealloc 6.27 1.90 0.24 PyObject_Call 5.74 2.12 0.22 PyTuple_New 5.48 2.33 0.21 property_descr_get 5.22 2.53 0.20 _PyType_Lookup 4.44 2.70 0.17 tuplesubscript 4.18 2.86 0.16 PyArg_UnpackTuple 3.92 3.01 0.15 PyNumber_AsSsize_t 3.66 3.15 0.14 PyObject_GetItem 2.87 3.26 0.11 itemgetter_call 2.61 3.36 0.10 PyObject_GC_UnTrack 1.70 3.43 0.07 PyObject_GetAttr 1.31 3.48 0.05 repeat_next 1.31 3.53 0.05 repeat_traverse 1.04 3.57 0.04 attrgetter_new 1.04 3.61 0.04 property_descr_set 0.78 3.64 0.03 PyErr_Occurred 0.78 3.67 0.03 PyErr_Restore 0.78 3.70 0.03 PyLong_AsSsize_t 0.78 3.73 0.03 PyType_GetSlot 0.52 3.75 0.02 PyObject_GC_Track 0.26 3.76 0.01 PyArg_ValidateKeywordArguments 0.26 3.77 0.01
[issue24014] Second pass of PyCode_Optimize
New submission from Joe Jevnik: There are a lot of optimizations that are being missed by only running a single pass of PyCode_Optimize. I originally started by trying to optimize for De Morgan's Laws in if tests; however, I realized that the issue actually went away if you run the optimizer twice. I imagine that this would provide a lot of other performance increases because some of the patterns don't show up until the optimizer has already run once. For example: def f(a, b): ... if not a and not b: ... return True ... return False ... On default, this generates: dis(f) 2 0 LOAD_FAST0 (a) 3 UNARY_NOT 4 POP_JUMP_IF_FALSE 18 7 LOAD_FAST1 (b) 10 UNARY_NOT 11 POP_JUMP_IF_FALSE 18 3 14 LOAD_CONST 1 (True) 17 RETURN_VALUE 418 LOAD_CONST 2 (False) 21 RETURN_VALUE If we run the optimizer again, we can collapse some of the UNARY_NOT - POP_JUMP_IF_FALSE back into POP_JUMP_IF_TRUE, like this: dis(f) 2 0 LOAD_FAST0 (a) 3 POP_JUMP_IF_TRUE16 6 LOAD_FAST1 (b) 9 POP_JUMP_IF_TRUE16 3 12 LOAD_CONST 1 (True) 15 RETURN_VALUE 416 LOAD_CONST 2 (False) 19 RETURN_VALUE Also, Python/importlib.h blew up in the diff; should this be included in the patch? -- components: Interpreter Core files: secondpass.patch keywords: patch messages: 241626 nosy: ll priority: normal severity: normal status: open title: Second pass of PyCode_Optimize type: performance versions: Python 3.5 Added file: http://bugs.python.org/file39144/secondpass.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24014 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24014] Second pass of PyCode_Optimize
Joe Jevnik added the comment: I tried bumping the magic number; however, I still cannot get consistent results when running benchmark locally. I guess I will close this as I cannot consistently show an improvement. -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24014 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24014] Second pass of PyCode_Optimize
Joe Jevnik added the comment: I am actually getting inconsistent results from running benchmark; I am pretty surprised by this. I could look into making the second pass only happen if the code has the CO_OPTIMIZED bit set; however, based on the results I am seeing from the benchmark runs, I am not sure it is worth it. Does the benchmark tool recompile the code every run? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24014 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Joe Jevnik added the comment: This is a different case from raising an AttributeError inside the __call__; class C(object): ... def __call__(self): ... raise AttributeError() ... hasattr(C(), '__call__') True class D(object): ... @property ... def __call__(self): ... raise AttributeError() ... hasattr(C(), '__call__') False AttributeError was picked very intentionally for the example. The docs show that n(args) == n.__call__(args) if n has a __call__; however, if a property raises an AttributeError, then it really does not have a __call__. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Changes by Joe Jevnik j...@quantopian.com: -- nosy: +ll ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Joe Jevnik added the comment: I am also confused by this; I would imagine that callable(obj) would respect the descriptor protocol. I have a proposed patch that would make this work with descriptors. -- keywords: +patch Added file: http://bugs.python.org/file39090/callable.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Changes by Joe Jevnik j...@quantopian.com: Removed file: http://bugs.python.org/file39090/callable.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Joe Jevnik added the comment: Oops, I messed up the test case; here is a fixed version (the class name was wrong). Just a note: all the existing test cases passed AND the one proposed in this thread. I understand that it is currently working as intended; however, the argument is that the intended behaviour should be changed. -- Added file: http://bugs.python.org/file39093/callable.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Joe Jevnik added the comment: I don't think that using a property to define a callable in the class is a bug; while it seems less ideal, I don't understand why it would be unsupported with callable when it executes correctly. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Joe Jevnik added the comment: As ionelmc mentioned, it does address the issue proposed originally and in the patch this is added as another test case for the callable function -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Joe Jevnik added the comment: I don't see how it is a bug that you can make __call__ an arbitrary descriptor as long as it returns a valid callable. if n.__call__ is a valid callable, why should it matter that it was looked up as a descriptor or as an instancemethod? -- type: enhancement - behavior versions: +Python 2.7, Python 3.2, Python 3.3, Python 3.4 -Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23990] Callable builtin doesn't respect descriptors
Joe Jevnik added the comment: The purpose of callable is to report whether an instance is callable or not I am totally with you so far until you get to: and that information is available on the instance's class, via the presence of __call__. I don't understand why this assumption must be made. The following class is totally valid and callable (in the sense that I can use function call syntax on instances): class C(object): @property def __call__(self): return lambda: None Also, I don't understand why you would mention __iter__, __iter__ respects the descriptor protocol also: class C(object): ... @property ... def __iter__(self): ... return lambda: iter(range(10)) ... it = iter(C()) next(it) 0 next(it) 1 next(it) 2 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23990 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: I can look into rewriting the framework to use multiprocessing; however, should we split this into a separate issue or block this one until that work is done. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22189] collections.UserString missing some str methods
Joe Jevnik added the comment: I have added a patch to add these to UserString. I also wrote a test case that would check the UserString, UserList, and UserDict's methods to make sure that new methods to str, list, or dict (or the removal of one of those methods from the User* version) will cause a test failure. -- keywords: +patch nosy: +ll Added file: http://bugs.python.org/file39054/userobjectmethods.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22189 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: I am not sure yet; I don't have access to a machine that can run the test. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: The tests need to be run with higher memory caps, this can be set with the -M flag, like python -m test -M 64G test_bigmem. I am under the impression that these tests don't get run all the time so we might want to run that suite. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: Has the test passed or is it still hung? Also, I guess this means that the tuple is over-allocating, maybe I need to revise the memory assumptions to account for the growth algorithm. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: I think that the idea is not to get as close to the limit as possible but just to hard cap the memory usage of the test suite so that it doesn't get oom-killed. twouters, does this sound correct? Also, I think that this means that the new decorator is reporting a proper size hint. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment: I am updating the patch to include an entry in Misc/NEWS. -- Added file: http://bugs.python.org/file38994/namedtuple-indexer-with-news.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23910 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22631] Feature Request CAN_RAW_FD_FRAMES
Joe Jevnik added the comment: I am not going to be able to write docs on this because I am not totally sure how you would use this; There is an entry in the docs for CAN_* so that might be enough. I have a line now saying that this particular flag is available as of 3.5 though. -- Added file: http://bugs.python.org/file38946/CAN_RAW_FD_FRAMES-update.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22631 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: I am now computing the size based on the system's int and pointer size. -- keywords: +patch nosy: +ll Added file: http://bugs.python.org/file38945/bigmem.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22631] Feature Request CAN_RAW_FD_FRAMES
Joe Jevnik added the comment: I tried to provide a summary of the Linux page on the topic and then said that it was fully documented there. -- Added file: http://bugs.python.org/file38957/CAN_RAW_FD_FRAMES-doc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22631 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: I have updated the test to make another test that does not have a __len__. -- Added file: http://bugs.python.org/file38952/bigmem-update.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't
Joe Jevnik added the comment: here is a patch and test case. I added an entry to Misc/NEWS. I tried to follow the format; however, let me know if I have done this incorrectly. -- keywords: +patch nosy: +ll Added file: http://bugs.python.org/file38972/skipitems.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23926 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22631] Feature Request CAN_RAW_FD_FRAMES
Joe Jevnik added the comment: So I think the patch could be applied to 3.4; I will look into moving the other back sometime this week. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22631 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected
Joe Jevnik added the comment: Steve, we talked about the possibility of having you run some of these tests on a machine with a large amount of ram; I am adding you to the nosy list. -- nosy: +steve.dower ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5438 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22631] Feature Request CAN_RAW_FD_FRAMES
Joe Jevnik added the comment: I have added a patch to update this with conditional compilation. -- keywords: +patch nosy: +ll Added file: http://bugs.python.org/file38934/CAN_RAW_FD_FRAMES.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22631 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22631] Feature Request CAN_RAW_FD_FRAMES
Joe Jevnik added the comment: To show where I got my sources: https://github.com/torvalds/linux/commit/e2d265d3b587f5f6f8febc0222aace93302ff0be There does not appear to be any new structures needed other than supporting the constant. -- nosy: +larry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22631 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment: sorry, I meant pypy -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23910 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment:
# Original version / new python implementation
./python -m timeit -s from collections import namedtuple;a = namedtuple('a',
'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.07 usec per loop
# C implementation
./python -m timeit -s from collections import namedtuple;a = namedtuple('a',
'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.028 usec per loop
The fallback is the same implementation that is currently used so this should
have no affect on pypi.
--
Added file: http://bugs.python.org/file38905/namedtuple-indexer-update.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23910] C implementation of namedtuple (WIP)
Joe Jevnik added the comment: I stripped down the patch to only the descriptor like we had discussed. -- Added file: http://bugs.python.org/file38896/namedtuple.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23910 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
