[issue37270] Manage memory lifetime for all type-related objects.

2019-06-13 Thread Joe Jevnik


Change by Joe Jevnik :


--
keywords: +patch
pull_requests: +13925
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14066

___
Python tracker 
<https://bugs.python.org/issue37270>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37270] Manage memory lifetime for all type-related objects.

2019-06-13 Thread Joe Jevnik


New submission from Joe Jevnik :

When using PyType_FromSpec, the memory for PyType_Spec.name, Py_tp_methods, and 
Py_tp_members needs to somehow outlive the resulting type. This makes it hard 
to use this interface to generate types without just leaking the memory for 
these arrays, which is bad if you are programatically creating short-lived 
types.

I posted about this on capi-sig, and the response seemed to be that it would be 
to replace the things that currently hold pointers into the array with copies 
of the data.

Remove internal usages of PyMethodDef and PyGetSetDef.

For PyMethodDef, change PyCFunctionObject to replace the PyMethodDef* member
with a PyCFunctionBase member. The PyCFunctionBase is a new struct to hold the
managed values of a PyMethodDef. This type is shared between PyCFunction and the
various callable descriptor objects. A PyCFunctionBase is like a PyMethodDef but
replaces the char* members with PyObject* members.

For PyGetSetDef, inline the members on the resulting PyGetSetDescrObject,
replacing all char* members with PyObject* members. The memory for the closure
is *not* managed, adding support for that would likely require an API change and
can be done in a future change.

For the tp_name field, instead of setting it directly to the value of
PyType_Spec.name, set it to the result of PyUnicode_AsUTF8(ht_name), where
ht_name is the PyUnicode object created from the original spec name. This is the
same trick used to properly manage this pointer for heap types when the __name__
is reassigned.

--
components: Extension Modules
messages: 345539
nosy: ll
priority: normal
severity: normal
status: open
title: Manage memory lifetime for all type-related objects.
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue37270>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36068] Make _tuplegetter objects serializable

2019-02-21 Thread Joe Jevnik


Joe Jevnik  added the comment:

Thank you for reviewing this so quickly!

--

___
Python tracker 
<https://bugs.python.org/issue36068>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36068] Make _tuplegetter objects serializable

2019-02-21 Thread Joe Jevnik


Change by Joe Jevnik :


--
keywords: +patch
pull_requests: +12006
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue36068>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36068] Make _tuplegetter objects serializable

2019-02-21 Thread Joe Jevnik


New submission from Joe Jevnik :

The new _tuplegetter objects for accessing fields of a namedtuple are no longer 
serializable with pickle. Cloudpickle, a library which provides extensions to 
pickle to facilitate distributed computing in Python, depended on being able to 
pickle the members of namedtuple classes. While property isn't serializable, 
cloudpickle has support for properties allowing us to serialize the old 
property(itemgetter) members.

The attached PR adds a __reduce__ method to _tuplegetter objects which will 
allow serialization without special support. Another option would be to expose 
`index` as a read-only attribute, allowing cloudpickle or other libraries to 
provide the pickle implementation as a third-party library.

--
components: Library (Lib)
messages: 336251
nosy: ll
priority: normal
severity: normal
status: open
title: Make _tuplegetter objects serializable
type: enhancement
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue36068>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23927] getargs.c skipitem() doesn't skip 'w*'

2018-07-23 Thread Joe Jevnik


Change by Joe Jevnik :


--
pull_requests: +7946

___
Python tracker 
<https://bugs.python.org/issue23927>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't

2018-07-09 Thread Joe Jevnik


Change by Joe Jevnik :


--
pull_requests: +7754

___
Python tracker 
<https://bugs.python.org/issue23926>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28638] Optimize namedtuple creation

2017-07-18 Thread Joe Jevnik

Joe Jevnik added the comment:

I added a benchmark suite (using Victor's perf utility) to cnamedtuple. The 
results are here: https://github.com/ll/cnamedtuple#benchmarks

To summarize: type creation is much faster; instance creation and named 
attribute access are a bit faster.

--
nosy: +ll

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28638>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13349] Non-informative error message in index() and remove() functions

2017-05-27 Thread Joe Jevnik

Changes by Joe Jevnik <j...@quantopian.com>:


--
pull_requests: +1919

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue13349>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30477] tuple.index error message improvement

2017-05-25 Thread Joe Jevnik

Joe Jevnik added the comment:

As a more meta question: I have noticed that many error messages in the stdlib 
use PyErr_SetString, or choose to use the type name instead of the repr of the 
object. Is there a reason for this? I normally try to fill the error message 
with as much context as possible to make debugging easier. Is it a performance 
worry?

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30477>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30477] tuple.index error message improvement

2017-05-25 Thread Joe Jevnik

Joe Jevnik added the comment:

I agree, "%R in tuple" is enough information for me. This would also remove the 
need to manually repr the object.

--
type: enhancement -> 

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30477>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30477] tuple.index error message improvement

2017-05-25 Thread Joe Jevnik

New submission from Joe Jevnik:

The old error of tuple.index(x): x not in tuple seemed very confusing to me. It 
was also harder to quickly understand what the program was doing wrong. This 
saves people a second pass through the program under the debugger because they 
can just see what the invalid value was.

The reason I am explicitly calling repr instead of using %R is that I didn't 
want to call repr twice if I didn't need to. If people think that is fine the 
format can just be tuple.index(%R): %R not in tuple instead.

--
messages: 294516
nosy: brett.cannon, ll
priority: normal
pull_requests: 1906
severity: normal
status: open
title: tuple.index error message improvement
versions: Python 3.7

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30477>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30180] PyArg_ParseTupleAndKeywords supports required keyword only arguments

2017-04-26 Thread Joe Jevnik

New submission from Joe Jevnik:

I opened a pr to remove a line in the docs about $ needing to follow | in 
PyArg_ParseTupleAndKeywords. In practice, you can just use a $ to create 
required keyword arguments which intuitively makes sense. I was told this 
should raise a SystemError; however, you can create required keyword only 
arguments in Python so I am not sure why we would want to fail when this is 
done with PyArg_ParseTupleAndKeywords.

--
messages: 292385
nosy: ll, serhiy.storchaka
priority: normal
pull_requests: 1417
severity: normal
status: open
title: PyArg_ParseTupleAndKeywords supports required keyword only arguments

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30180>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue30180] PyArg_ParseTupleAndKeywords supports required keyword only arguments

2017-04-26 Thread Joe Jevnik

Changes by Joe Jevnik <j...@quantopian.com>:


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30180>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29468] zipfile should catch ValueError as well as OSError to detect bad seek calls

2017-02-06 Thread Joe Jevnik

New submission from Joe Jevnik:

In zipfile.py only OSError is checked to see if seek fails:

```
def _EndRecData64(fpin, offset, endrec):
"""
Read the ZIP64 end-of-archive records and use that to update endrec
"""
try:
fpin.seek(offset - sizeEndCentDir64Locator, 2)
except OSError:
# If the seek fails, the file is not large enough to contain a ZIP64
# end-of-archive record, so just return the end record we were given.
return endrec
```

I belive that this should also catch ValueError so that other file-like objects 
may be passed to ZipFile. The particular case I ran into was passing an mmap 
object:

```
"""
$ python p.py
sys.version_info(major=3, minor=6, micro=0, releaselevel='final', serial=0)
[]
Traceback (most recent call last):
  File "p.py", line 34, in 
with mmap_shared_raw_zipfile(f.name) as zf:
  File "/usr/lib64/python3.6/contextlib.py", line 82, in __enter__
return next(self.gen)
  File "p.py", line 23, in mmap_shared_raw_zipfile
ZipFile(mm) as zf:
  File "/usr/lib64/python3.6/zipfile.py", line 1100, in __init__
self._RealGetContents()
  File "/usr/lib64/python3.6/zipfile.py", line 1163, in _RealGetContents
endrec = _EndRecData(fp)
  File "/usr/lib64/python3.6/zipfile.py", line 264, in _EndRecData
return _EndRecData64(fpin, -sizeEndCentDir, endrec)
  File "/usr/lib64/python3.6/zipfile.py", line 196, in _EndRecData64
fpin.seek(offset - sizeEndCentDir64Locator, 2)
ValueError: seek out of range
"""
from contextlib import contextmanager
import mmap
import sys
from tempfile import NamedTemporaryFile
from zipfile import ZipFile


print(sys.version_info)


@contextmanager
def mmap_shared_raw_zipfile(path):
"""Open a zipfile with mmap shared so the data can be shared in multiple
processes.

Parameters
--
path : str
The path the zipfile to open.

Notes
-
The context manager returns a :class:`zipfile.ZipFile` on enter.
"""
with open(path) as f, \
mmap.mmap(f.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ) as mm, \
ZipFile(mm) as zf:
yield zf


with NamedTemporaryFile() as f:
ZipFile(f, mode='w').close()
print(ZipFile(f.name).infolist())


with NamedTemporaryFile() as f:
ZipFile(f, mode='w').close()
with mmap_shared_raw_zipfile(f.name) as zf:
print(zf.infolist())
```

--
components: Library (Lib)
messages: 287185
nosy: ll
priority: normal
severity: normal
status: open
title: zipfile should catch ValueError as well as OSError to detect bad seek 
calls
versions: Python 3.6

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29468>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29096] Signal Handlers reliably cause UnboundLocalErrors

2016-12-28 Thread Joe Jevnik

Joe Jevnik added the comment:

The issue appears to be in ceval.c:unicode_concatenate (or the py2 equivalent)

The code sets the local variable on the lhs to NULL before doing a potentially 
inplace append to the string. This means that if a signal is raised during the 
concat, we never hit the STORE_FAST instruction following the INPLACE_ADD so 
the local or cell still holds NULL, triggering an error.

I am not really sure what can be done to prevent the failure while still 
allowing the optimization.

--
components: +Interpreter Core
nosy: +ll
type: crash -> 

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29096>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled

2016-09-27 Thread Joe Jevnik

Joe Jevnik added the comment:

I definitely could have used PyImport_Import and PyObject_Call to accomplish 
this task; however, when I looked at at the implementation of these functions 
it seemed like a lot of overhead and book keeping just to set a boolean. Since 
I was already in a C extension it would be nicer to not need to worry about 
PyObjects and ref counts just to set this value so I thought it would be nice 
to expose these small functions to users. Since this is equivalent to gc.enable 
or gc.disable I don't think there is a deep can of worms here. The only 
difference in the implementation is that it doesn't return None. There is 
already PyGC_Collect, so I figured these were just omitted because no one 
thought to add them. I don't have a burning desire to get this in, I just 
thought it would be a nice to have.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28254>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue28254] Add C API for gc.enable, gc.disable, and gc.isenabled

2016-09-23 Thread Joe Jevnik

New submission from Joe Jevnik:

I was writing an extension module that was working with weakrefs and wanted to 
ensure that the GC would not run for a block of code. I noticed that 
gc.enable/gc.disable are not exposed to C and the state of the gc is in a 
static variable so it cannot be mutated from an extension.

This change adds an easier way to access this functionality in the C api 
without going through an import or PyObject_Call.

I am wondering where to document these functions as well as PyGC_Collect. I 
didn't see that function anywhere in the docs (and didn't know about it) so I 
would like to make them more visible. My first thought was 
Doc/c-api/gcsupport.rst but this is not in service of supporting the GC, it is 
about the state of the GC itself. If that seems like a decent place I could add 
a small section at the bottom about interacting with the GC and document these 
new functions and PyGC_Collect.

I'm sorry if this is exposed elsewhere. If it is I can try to add some links to 
i

--
components: Extension Modules
files: gc-capi.patch
keywords: patch
messages: 277245
nosy: ll
priority: normal
severity: normal
status: open
title: Add C API for gc.enable, gc.disable, and gc.isenabled
versions: Python 3.7
Added file: http://bugs.python.org/file44787/gc-capi.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue28254>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27241] Catch exceptions raised in pstats add (repl)

2016-08-02 Thread Joe Jevnik

Joe Jevnik added the comment:

bump

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27241>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27359] OrderedDict pseudo-literals (WIP)

2016-06-20 Thread Joe Jevnik

Joe Jevnik added the comment:

Sorry about undoing the settings, that was unintentional. The UI around these 
settings is sort of unintuitive.

I was waiting to propose this on python-dev until I got it working, but I 
wanted to push up what I did have as a work in progress so I could try to get 
some help with the crash I am seeing. In the future I will upload the patch 
somewhere else until it is done.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27359>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27359] OrderedDict pseudo-literals (WIP)

2016-06-20 Thread Joe Jevnik

Joe Jevnik added the comment:

Assuming all dicts are ordered, there will be no need for this (as long as the 
parser preserves order) so I am okay with dropping this.

One of the main use cases I have for wanting nice literals is when I am working 
on tests. I find it very nice to be able to write out the expected results in a 
clear way. While the app code might not be using the literals, it improves the 
readability of the tests, which make it easier to see that the test is correct.

--
resolution: later -> 
status: closed -> open

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27359>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27359] OrderedDict pseudo-literals (WIP)

2016-06-20 Thread Joe Jevnik

New submission from Joe Jevnik:

This proposes the following syntax for creating OrderedDict literals:

OrderedDict[k1: v1, k2: v2, ...]

This is implemented by putting a metaclass on OrderedDict which has a getitem 
that turns the slices into a list of tuples (after validation).

The idea is taken from the library datashape where we have done this to 
implement pseudo literals for the Record type.

This is marked as  WIP because I am not quite sure why, but this patch is 
causing subclasses of OrderedDict to segfault in PyObject_SetItem.

--
components: Library (Lib)
files: odict_literals.patch
keywords: patch
messages: 268954
nosy: ll
priority: normal
severity: normal
status: open
title: OrderedDict pseudo-literals (WIP)
versions: Python 3.6
Added file: http://bugs.python.org/file43495/odict_literals.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27359>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27241] Catch exceptions raised in pstats add (repl)

2016-06-06 Thread Joe Jevnik

New submission from Joe Jevnik:

I was trying to add a file and accidently mistyped the name which crashed the 
repl session. I think it would be nicer to report the failure but bring me back 
to the prompt so I can decide what I would like to do.

This patch catches any IOErrors that can be raised when adding a file in the 
repl, reports it to the user, and then returns them to the repl.

--
components: Demos and Tools
files: catch-pstats-add.patch
keywords: patch
messages: 267542
nosy: ll
priority: normal
severity: normal
status: open
title: Catch exceptions raised in pstats add (repl)
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file43262/catch-pstats-add.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27241>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27039] bytearray.remove cannot remove bytes with value greater than 127

2016-05-16 Thread Joe Jevnik

Joe Jevnik added the comment:

This seems to just be a bug in the implementation of remove. I have a patch to 
fix this and a test case.

--
keywords: +patch
nosy: +ll
Added file: http://bugs.python.org/file42875/bytearray-remove.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27039>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2016-05-04 Thread Joe Jevnik

Joe Jevnik added the comment:

people have had some time to think about this, whats should the name be so we 
can merge this?

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26802] Avoid copy in call_function_var when no extra stack args are passed

2016-04-19 Thread Joe Jevnik

Joe Jevnik added the comment:

CALL_FUNCTION doesn't use extended arg; I don't see what we get by adding some 
opcode to convert the sequence into a tuple. We also don't want to box up the 
stack arguments into a tuple because when we do a CALL_FUNCTION to a python 
function we just copy the stack[-n:] elements into the locals of the next 
frame. Emitting a build_tuple here would be an unneeded allocation.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26802] Avoid copy in call_function_var when no extra stack args are passed

2016-04-19 Thread Joe Jevnik

Joe Jevnik added the comment:

> It's possible that stararg is already an empty tuple. Is it worth to use it?

PyTuple_New(0) should return the unit tuple in most cases:

#if PyTuple_MAXSAVESIZE > 0
if (size == 0 && free_list[0]) {
op = free_list[0];
Py_INCREF(op);
#ifdef COUNT_ALLOCS
tuple_zero_allocs++;
#endif
return (PyObject *) op;
}


Looking at this again I think that checking if stararg is nonnull is more 
clear, I will update the patch (as call-function-var-3.patch). I cannot exactly 
rewrite the if to use the control flow you show because that would cause 
non-tuple subclasses to forward a stararg of () instead of copying it into a 
normal tuple when no arguments are passed ont the stack.

--
Added file: http://bugs.python.org/file42521/call-function-var-3.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26802] Avoid copy in call_function_var when no extra stack args are passed

2016-04-19 Thread Joe Jevnik

Changes by Joe Jevnik <j...@quantopian.com>:


--
type:  -> performance

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26802] Avoid copy in call_function_var when no extra stack args are passed

2016-04-19 Thread Joe Jevnik

Joe Jevnik added the comment:

in _PyEval_EvalCodeWithName we will have already pulled the underlying array 
out of the tuple subclass and the then we will reconstruct the vararg value in 
the locals from this array. This means that with this patch we can get:

>>> class C(tuple): pass
... 
>>> def f(*args): print(type(args))
... 
>>> f(*C([1, 2, 3]))


The only case where we get potentially strange behavior is if we have a type 
whose tp_call did not forward to PyArg_Parse* or the PyTupleObject api and went 
though the abstract object interface. I ran the tests and the benchmark suite 
and did not run into any issues here but I can see how this would be 
potentially problematic behavior.

I have updated the code to also do a PyTuple_CheckExact against `stararg`. The 
tests I ran are:

timeit("h(*(a, b, c, d, e, f, g))", 'def h(a, b, c, d, e, f, g): pass\na, b, c, 
d, e, f, g = range(7)', number=int(10e7))

default: 17.191688070015516
patch: 14.060781285981648

There is also a really great case where you star unpack a tuple of literals 
which gets pushed into the co_consts:

timeit("f(*('a', 'b', 'c', 'd', 'e', 'f', 'g'))", 'def f(a, b, c, d, e, f, g): 
pass', number=int(10e7))

default: 13.842308042978402
patch: 9.696972723002546

This case seems far less common, but maybe someone has something like:

a = 1, 2, 3
...
f(*a)

--
type: performance -> 
Added file: http://bugs.python.org/file42515/call-function-var.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't

2016-04-18 Thread Joe Jevnik

Joe Jevnik added the comment:

bump

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue23926>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26802] Avoid copy in call_function_var when no extra stack args are passed

2016-04-18 Thread Joe Jevnik

New submission from Joe Jevnik:

When star unpacking positions into a function we can avoid a copy iff there are 
no extra arguments coming from the stack. Syntactically this looks like: 
`f(*args)`

This reduces the overhead of the call by about 25% (~30ns on my machine) based 
on some light profiling of just * unpacking. I can imagine this having a slight 
impact on code that uses this feature in a loop.

The cost is minimal when we need to make the copy because the int != 0 check is 
dwarfed by the allocation. I did not notice a significant change between the 
slow path and the original code.

The macro benchmark suite did not show a difference on my machine but this is 
not much more complex code.

--
components: Interpreter Core
files: call-function-var.patch
keywords: patch
messages: 263702
nosy: ll
priority: normal
severity: normal
status: open
title: Avoid copy in call_function_var when no extra stack args are passed
versions: Python 3.6
Added file: http://bugs.python.org/file42512/call-function-var.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26802>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26448] dis.findlabels ignores EXTENDED_ARG

2016-02-26 Thread Joe Jevnik

Changes by Joe Jevnik <j...@quantopian.com>:


--
nosy: +ll

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26448>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26379] zlib decompress as_bytearray flag

2016-02-18 Thread Joe Jevnik

Joe Jevnik added the comment:

The recipe you showed looks like it is very complicated for the expected use 
case of decompressing all of the data into a mutable buffer. One difference I 
see with this and struct.pack_into or socket.recv_into is that in both of those 
cases we know how large our buffer needs to be. With zlib.decompress there is 
not a simple way for users to preallocate a buffer to hold the result and need 
to resort to some looping with a lot of resizes. I think that returning either 
a mutable or immutable buffer hits 95% of use cases. If we go with the 
decompress_into method I would strongly advise putting decompress_as_bytearray 
into the standard library so that users do not need write this themselves.

Thank you for pointing me at the bz2 and LZMA modules. When we come to an 
agreement on the API I will update bz2 and LZMA so that they match. Should I 
also update the gzip module?

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26379>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2016-02-17 Thread Joe Jevnik

Joe Jevnik added the comment:

ping: any decision on this?

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue26379] zlib decompress as_bytearray flag

2016-02-17 Thread Joe Jevnik

New submission from Joe Jevnik:

Adds a keyword only flag to zlib.decompress and zlib.Decompress.decompress 
which marks that the return type should be a bytearray instead of bytes.

The use case for this is reading compressed data into a mutable array to do 
further operations without requiring that the user copy the data first. Often, 
if a user is choosing to compress the data there might be a real cost to 
copying the uncompressed bytes into a mutable buffer.

The impetus for this change comes from a patch for Pandas 
(https://github.com/pydata/pandas/pull/12359). I have also worked on a similar 
feature for blosc, another popular compression library for python 
(https://github.com/Blosc/python-blosc/pull/107).

My hope is to fix the hacky solution presented in the pandas patch by 
supporting this feature directly in the compression libraries.

As a side note: this is my first time using the argument clinic and I love it. 
It was so simple to add this argument, thank you very much for developing such 
an amazing tool!

--
components: Extension Modules
files: zlib.patch
keywords: patch
messages: 260424
nosy: ll
priority: normal
severity: normal
status: open
title: zlib decompress as_bytearray flag
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file41944/zlib.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26379>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-12-22 Thread Joe Jevnik

Joe Jevnik added the comment:

Is there a decision on the name? I can update the patch if needed.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-12-22 Thread Joe Jevnik

Joe Jevnik added the comment:

> and the latter might give the impression it was some sort of special 
> method/attribute when it was not.

Wouldn't adding this be special because it is specifically reserved by CPython 
as an implementation detail?

Also, would adding this name (__stub or __stub__) be sufficient to get this 
patch merged again. I would not mind investigating the gc_head issue after but 
I view that as a separate issue which is only exercised by this patch.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue24379>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-12-02 Thread Joe Jevnik

Joe Jevnik added the comment:

partial's unique usage is why I feel like it would be so jarring for them do be 
named differently. I would be happiest having this feature at all so I will 
change the name to 'kwargs' if you would like. I just want to be sure that my 
reasons for choosing this name in the first place ere understood.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-12-01 Thread Joe Jevnik

Joe Jevnik added the comment:

Thanks for pointing me at the refleak, uploading an update

--
Added file: http://bugs.python.org/file41203/methodcaller-attrs-1.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-12-01 Thread Joe Jevnik

Joe Jevnik added the comment:

Added a test case for the mutation of keywords.

--
Added file: http://bugs.python.org/file41204/methodcaller-attrs-2.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-12-01 Thread Joe Jevnik

Joe Jevnik added the comment:

I agree that it is a strange name and I also think that it could be immutable 
or a copy of the internal dict; however, I think that consistency with existing 
APIs in the standard library is more important. 'keywords' is still very clear 
in context and is used in 'functools.partial'. This feature is very similar to 
'partial' so I would like to follow the example set there.

I would not mind changing 'partial' to reflect this feedback; but, I imagine 
that will break people's code and be a harder sell. I would find it frustrating 
if 'partial' and 'methodcaller' spelled 'keywords' differently or had slightly 
different behavior when it comes to the keyword argument dictionary. Allowing 
the mutation is not that unintuitive if you look at the simple python 
translation I added to the docs.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-12-01 Thread Joe Jevnik

Joe Jevnik added the comment:

I only want this feature to keep the usage close to functools.partial. I was 
actually sort of surprised to see that mutation of the dict held in partial, 
but I would rather be consistent.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-12-01 Thread Joe Jevnik

Joe Jevnik added the comment:

Thanks for review, looking into the reference count issue.

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25770] expose name, args, and kwargs from methodcaller

2015-11-30 Thread Joe Jevnik

New submission from Joe Jevnik:

This patch adds 3 properties to methodcaller objects for inspecting the object 
at runtime:

1. 'name': the name of the method to call
2. 'args': the position arguments to pass to the method
3. 'keywords': the keyword arguments to pass to the method

args and keywords act like functools.partial (that is why I did not name it 
kwargs).

I noticed that recently the repr changed to expose this information which helps 
in the debugging use case; however, this allows us to use the methodcaller to 
maybe construct a new methodcaller with different args, or call a different 
method with the same args.

--
components: Library (Lib)
files: methodcaller-attrs.patch
keywords: patch
messages: 255636
nosy: ll
priority: normal
severity: normal
status: open
title: expose name, args, and kwargs from methodcaller
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file41198/methodcaller-attrs.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25770>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't

2015-10-20 Thread Joe Jevnik

Joe Jevnik added the comment:

bumping this issue

--

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue23926>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-08-16 Thread Joe Jevnik

Joe Jevnik added the comment:

Sorry about the ideas thread. Thank you for merging this!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-28 Thread Joe Jevnik

Joe Jevnik added the comment:

I posted this to python Ideas on June 10th of this year. That is where we 
decided to rename it from `slice.literal` to `operator.subscript`.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't

2015-07-25 Thread Joe Jevnik

Joe Jevnik added the comment:

bumping so that we don't forget about this.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23926
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-24 Thread Joe Jevnik

Joe Jevnik added the comment:

Any more comments?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't

2015-07-15 Thread Joe Jevnik

Joe Jevnik added the comment:

Sorry it took so long to get back to this. I didn't realize this was still 
open. I have provided the update to the docs and moved it to the 3.6 section. I 
also made sure the patch still applies and the tests all pass.

--
Added file: http://bugs.python.org/file39931/skipitems.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23926
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-14 Thread Joe Jevnik

Joe Jevnik added the comment:

I updated the docstring

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-12 Thread Joe Jevnik

Joe Jevnik added the comment:

updating to address the docs and order of assertions

--
Added file: http://bugs.python.org/file39904/operator_subscript_pyonly.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-12 Thread Joe Jevnik

Joe Jevnik added the comment:

is it normal to get a lot of 500s when using the review system?

--
Added file: http://bugs.python.org/file39906/operator_subscript_pyonly.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-12 Thread Joe Jevnik

Changes by Joe Jevnik j...@quantopian.com:


Added file: http://bugs.python.org/file39907/operator_subscript_pyonly.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-12 Thread Joe Jevnik

Changes by Joe Jevnik j...@quantopian.com:


Added file: http://bugs.python.org/file39913/operator_subscript_pyonly.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-11 Thread Joe Jevnik

Joe Jevnik added the comment:

Ah, I hadn't seen that, I will address those now, sorry about that.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-11 Thread Joe Jevnik

Joe Jevnik added the comment:

updating with the slots change

This also adds a ton of test cases

--
Added file: http://bugs.python.org/file39902/operator_subscript_pyonly.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-07-11 Thread Joe Jevnik

Joe Jevnik added the comment:

ping: is there anything blocking this?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-06-23 Thread Joe Jevnik

Joe Jevnik added the comment:

I removed the C implementation.

--
Added file: http://bugs.python.org/file39792/operator_subscript_pyonly.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-06-22 Thread Joe Jevnik

Joe Jevnik added the comment:

Based on some discussion on python-ideas, this is being renamed to 
operator.subscript. Here is the patch that makes the correct the changes to 
move this object into the operator module.

--
components: +Extension Modules -Interpreter Core
title: slice.literal notation - operator.subscript
Added file: http://bugs.python.org/file39775/operator_subscript.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] operator.subscript

2015-06-22 Thread Joe Jevnik

Joe Jevnik added the comment:

I just moved it over since I implemented it for slice originally, I can drop 
the C implementation.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] slice.literal notation

2015-06-10 Thread Joe Jevnik

Joe Jevnik added the comment:

It is a singleton, does not accept the `maketuple` flag, and is written in C. I 
did not know about the s_ attribute of numpy before writing this; however, I 
still think that moving this object to slice improves code clarity (s_ is not a 
super clear name). I also think that this behaviour belongs on the slice object.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] slice.literal notation

2015-06-10 Thread Joe Jevnik

Joe Jevnik added the comment:

 slice.literal[0]
0
 y = slice.literal[1:2]
slice(1, 2, None)
 slice.literal[0:1, ..., 3]
(slice(0, 1, None), Ellipsis, 3)

The way this object works right now does not create instances of some inner 
class of slice, instead, indexing it returns the key without modification.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24379] slice.literal notation

2015-06-04 Thread Joe Jevnik

Joe Jevnik added the comment:

 Why not index the slice type itself? slice[1:2]

I originally considered this and I personally really like this syntax, but I 
was concerned with ambiguity with the typing module

 The only question in my mind is what slice should do when given just a single 
 index

I think some of the power of this concept comes from the fact that I can 
express a complicated indexer without worrying about how it desugars. I would 
personally prefer being able to have this return tuples and scalars so that the 
syntax is easier to explain.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24379
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] property_descr_get reuse argument tuple

2015-04-29 Thread Joe Jevnik

Joe Jevnik added the comment:

I don't think that we need to worry about reusing the single argument tuple in 
a recursive situation because we never need the value after we start the call. 
We also just write our new value and then clean up with a NULL to make sure 
that we don't blow up when we dealloc the tuple. For example:

 class C(object):
... @property
... def f(self):
... return D().f
... 
 class D(object):
... @property
... def f(self):
... return 1
... 
 C().f
1


This works with recursive properties.
I also think that is is getting cleaned up on shutdown, if I put a pointer to 
garbage in the tuple, the interpreter blows up on shutdown; this makes me think 
that tuple_dealloc is being called somewhere. About putting the tuple on the 
property instance, that would nice for memory management; however, that 
increases the memory overhead of each property. This also means that we would 
only get the faster lookup after the property has been accessed once; this is 
fine but the current implementation makes it so that all properties are faster 
after any of them are looked up once. I could be wrong about the cleanup though.

I am also updating the title and headers because this issue is no longer about 
namedtuple.

--
components: +Interpreter Core -Extension Modules
title: C implementation  of namedtuple (WIP) - property_descr_get reuse 
argument tuple

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-27 Thread Joe Jevnik

Joe Jevnik added the comment:

I switched to the static tuple.

--
Added file: http://bugs.python.org/file39216/with-static-tuple.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-27 Thread Joe Jevnik

Joe Jevnik added the comment:

I don't think that I can cache the __call__ of the fget object because it might 
be an instance of a heaptype, and if someone changed the __class__ of the 
object in between calls to another heaptype that had a different __call__, you 
would still get the __call__ from the first type. I also don't know if this is 
supported behavior or just something that works by accident.

I read through PyObject_Call, and all the code is needed assuming we are not 
caching the tp_call value.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-27 Thread Joe Jevnik

Joe Jevnik added the comment:

I am currently on a different machine so these numbers are not relative to the 
others posted earlier.

* default
./python -m timeit -s from collections import namedtuple as n;a = n('n', 'a b 
c')(1, 2, 3) a.a
1000 loops, best of 3: 0.0699 usec per loop

* patch
./python -m timeit -s from collections import namedtuple as n;a = n('n', 'a b 
c')(1, 2, 3) a.a
1000 loops, best of 3: 0.0468 usec per loop

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-26 Thread Joe Jevnik

Joe Jevnik added the comment:

I was unable to see a performance increase by playing with the 
itemgetter.__call__ code; however, updating the propery code seemed to show a 
small improvement. I think that for simple indexing the cost of checking if it 
is a sequence outways the faster dispatch (when using PySequence_GetItem). I 
can play with this further.

* default
[joejev@Sheila cpython]$ ./python -m timeit -s from collections import 
namedtuple as n;a = n('n', 'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.101 usec per loop

* patch
[joejev@Sheila cpython]$ ./python -m timeit -s from collections import 
namedtuple as n;a = n('n', 'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.0942 usec per loop

--
Added file: http://bugs.python.org/file39210/property.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-26 Thread Joe Jevnik

Joe Jevnik added the comment:

This was very exciting, I have never run gprof before; so just to make sure I 
did this correctly, I will list my steps:

1. compile with the -pg flag set
1. run the test with ./python -m timeit ...
1. $ gprof python gmon.out  profile.out

Here is default:

Each sample counts as 0.01 seconds.
  %   cumulative   self  self total   
 time   seconds   secondscalls  Ts/call  Ts/call  name
 22.48  0.87 0.87 PyEval_EvalFrameEx
  9.82  1.25 0.38 
PyObject_CallFunctionObjArgs
  6.85  1.52 0.27 PyObject_GenericGetAttr
  6.46  1.77 0.25 tupledealloc
  5.56  1.98 0.22 PyArg_UnpackTuple
  5.17  2.18 0.20 PyNumber_AsSsize_t
  5.17  2.38 0.20 tuplesubscript
  5.04  2.58 0.20 PyObject_Call
  4.91  2.77 0.19 _PyType_Lookup
  4.65  2.95 0.18 PyTuple_New
  4.65  3.13 0.18 PyObject_GetItem
  4.39  3.30 0.17 itemgetter_call
  1.94  3.37 0.08 PyObject_GetAttr
  1.81  3.44 0.07 PyObject_GC_UnTrack
  1.81  3.51 0.07 _PyTuple_DebugMallocStats
  1.03  3.55 0.04 PyErr_Occurred
  1.03  3.59 0.04 property_descr_set
  1.03  3.63 0.04 tuplerepr
  0.78  3.66 0.03 PyLong_AsSsize_t
  0.78  3.69 0.03 PyObject_GC_Track
  0.52  3.71 0.02 property_descr_get
  0.52  3.73 0.02 repeat_next
  0.52  3.75 0.02 repeat_traverse
  0.39  3.77 0.02 
PyArg_ValidateKeywordArguments
  0.39  3.78 0.02 _Py_CheckFunctionResult
  0.26  3.79 0.01 PyCFunction_NewEx
  0.26  3.80 0.01 PyCode_New
  0.26  3.81 0.01 PyErr_Restore
  0.26  3.82 0.01 PyType_GetSlot
  0.26  3.83 0.01 
_PyObject_CallMethodIdObjArgs
  0.26  3.84 0.01 attrgetter_new
  0.26  3.85 0.01 update_one_slot
  0.13  3.86 0.01 
_PyObject_GenericGetAttrWithDict
  0.13  3.86 0.01 _PyObject_SetAttrId
  0.13  3.87 0.01 gc_isenabled
  0.13  3.87 0.01 visit_decref


Here is my patch:

Each sample counts as 0.01 seconds.
  %   cumulative   self  self total   
 time   seconds   secondscalls  Ts/call  Ts/call  name
 26.63  1.02 1.02 PyEval_EvalFrameEx
  8.88  1.36 0.34 PyObject_GenericGetAttr
  7.83  1.66 0.30 tupledealloc
  6.27  1.90 0.24 PyObject_Call
  5.74  2.12 0.22 PyTuple_New
  5.48  2.33 0.21 property_descr_get
  5.22  2.53 0.20 _PyType_Lookup
  4.44  2.70 0.17 tuplesubscript
  4.18  2.86 0.16 PyArg_UnpackTuple
  3.92  3.01 0.15 PyNumber_AsSsize_t
  3.66  3.15 0.14 PyObject_GetItem
  2.87  3.26 0.11 itemgetter_call
  2.61  3.36 0.10 PyObject_GC_UnTrack
  1.70  3.43 0.07 PyObject_GetAttr
  1.31  3.48 0.05 repeat_next
  1.31  3.53 0.05 repeat_traverse
  1.04  3.57 0.04 attrgetter_new
  1.04  3.61 0.04 property_descr_set
  0.78  3.64 0.03 PyErr_Occurred
  0.78  3.67 0.03 PyErr_Restore
  0.78  3.70 0.03 PyLong_AsSsize_t
  0.78  3.73 0.03 PyType_GetSlot
  0.52  3.75 0.02 PyObject_GC_Track
  0.26  3.76 0.01 
PyArg_ValidateKeywordArguments
  0.26  3.77 0.01

[issue24014] Second pass of PyCode_Optimize

2015-04-20 Thread Joe Jevnik

New submission from Joe Jevnik:

There are a lot of optimizations that are being missed by only running a single 
pass of PyCode_Optimize. I originally started by trying to optimize for De 
Morgan's Laws in if tests; however, I realized that the issue actually went 
away if you run the optimizer twice. I imagine that this would provide a lot of 
other performance increases because some of the patterns don't show up until 
the optimizer has already run once. For example:

 def f(a, b):
... if not a and not b:
... return True
... return False
... 

On default, this generates:
 dis(f)
  2   0 LOAD_FAST0 (a)
  3 UNARY_NOT
  4 POP_JUMP_IF_FALSE   18
  7 LOAD_FAST1 (b)
 10 UNARY_NOT
 11 POP_JUMP_IF_FALSE   18

  3  14 LOAD_CONST   1 (True)
 17 RETURN_VALUE

  418 LOAD_CONST   2 (False)
 21 RETURN_VALUE


If we run the optimizer again, we can collapse some of the UNARY_NOT - 
POP_JUMP_IF_FALSE back into POP_JUMP_IF_TRUE, like this:
 dis(f)
  2   0 LOAD_FAST0 (a)
  3 POP_JUMP_IF_TRUE16
  6 LOAD_FAST1 (b)
  9 POP_JUMP_IF_TRUE16

  3  12 LOAD_CONST   1 (True)
 15 RETURN_VALUE

  416 LOAD_CONST   2 (False)
 19 RETURN_VALUE


Also, Python/importlib.h blew up in the diff; should this be included in the 
patch?

--
components: Interpreter Core
files: secondpass.patch
keywords: patch
messages: 241626
nosy: ll
priority: normal
severity: normal
status: open
title: Second pass of PyCode_Optimize
type: performance
versions: Python 3.5
Added file: http://bugs.python.org/file39144/secondpass.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24014
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24014] Second pass of PyCode_Optimize

2015-04-20 Thread Joe Jevnik

Joe Jevnik added the comment:

I tried bumping the magic number; however, I still cannot get consistent 
results when running benchmark locally. I guess I will close this as I cannot 
consistently show an improvement.

--
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24014
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue24014] Second pass of PyCode_Optimize

2015-04-20 Thread Joe Jevnik

Joe Jevnik added the comment:

I am actually getting inconsistent results from running benchmark; I am pretty 
surprised by this. I could look into making the second pass only happen if the 
code has the CO_OPTIMIZED bit set; however, based on the results I am seeing 
from the benchmark runs, I am not sure it is worth it. Does the benchmark tool 
recompile the code every run?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue24014
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-19 Thread Joe Jevnik

Joe Jevnik added the comment:

This is a different case from raising an AttributeError inside the __call__;


 class C(object):
... def __call__(self):
... raise AttributeError()
...
 hasattr(C(), '__call__')
True
 class D(object):
... @property
... def __call__(self):
... raise AttributeError()
...
 hasattr(C(), '__call__')
False

AttributeError was picked very intentionally for the example.

The docs show that n(args) == n.__call__(args) if n has a __call__; however, if 
a property raises an AttributeError, then it really does not have a __call__.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Changes by Joe Jevnik j...@quantopian.com:


--
nosy: +ll

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Joe Jevnik added the comment:

I am also confused by this; I would imagine that callable(obj) would respect 
the descriptor protocol.

I have a proposed patch that would make this work with descriptors.

--
keywords: +patch
Added file: http://bugs.python.org/file39090/callable.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Changes by Joe Jevnik j...@quantopian.com:


Removed file: http://bugs.python.org/file39090/callable.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Joe Jevnik added the comment:

Oops, I messed up the test case; here is a fixed version (the class name was 
wrong). Just a note: all the existing test cases passed AND the one proposed in 
this thread.

I understand that it is currently working as intended; however, the argument is 
that the intended behaviour should be changed.

--
Added file: http://bugs.python.org/file39093/callable.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Joe Jevnik added the comment:

I don't think that using a property to define a callable in the class is a bug; 
while it seems less ideal, I don't understand why it would be unsupported with 
callable when it executes correctly.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Joe Jevnik added the comment:

As ionelmc mentioned, it does address the issue proposed originally and in the 
patch this is added as another test case for the callable function

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Joe Jevnik added the comment:

I don't see how it is a bug that you can make __call__ an arbitrary descriptor 
as long as it returns a valid callable. if n.__call__ is a valid callable, why 
should it matter that it was looked up as a descriptor or as an instancemethod?

--
type: enhancement - behavior
versions: +Python 2.7, Python 3.2, Python 3.3, Python 3.4 -Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23990] Callable builtin doesn't respect descriptors

2015-04-17 Thread Joe Jevnik

Joe Jevnik added the comment:

The purpose of callable is to report whether an instance is callable or not

I am totally with you so far until you get to: and that information is 
available on the instance's class, via the presence of __call__. I don't 
understand why this assumption must be made. The following class is totally 
valid and callable (in the sense that I can use function call syntax on 
instances):

class C(object):
@property
def __call__(self):
return lambda: None


Also, I don't understand why you would mention __iter__, __iter__ respects the 
descriptor protocol also:


 class C(object):
... @property
... def __iter__(self):
... return lambda: iter(range(10))
... 
 it = iter(C())
 next(it)
0
 next(it)
1
 next(it)
2

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23990
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-15 Thread Joe Jevnik

Joe Jevnik added the comment:

I can look into rewriting the framework to use multiprocessing; however, should 
we split this into a separate issue or block this one until that work is done.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22189] collections.UserString missing some str methods

2015-04-15 Thread Joe Jevnik

Joe Jevnik added the comment:

I have added a patch to add these to UserString. I also wrote a test case that 
would check the UserString, UserList, and UserDict's methods to make sure that 
new methods to str, list, or dict (or the removal of one of those methods from 
the User* version) will cause a test failure.

--
keywords: +patch
nosy: +ll
Added file: http://bugs.python.org/file39054/userobjectmethods.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22189
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-14 Thread Joe Jevnik

Joe Jevnik added the comment:

I am not sure yet; I don't have access to a machine that can run the test.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-14 Thread Joe Jevnik

Joe Jevnik added the comment:

The tests need to be run with higher memory caps, this can be set with the -M 
flag, like python -m test -M 64G test_bigmem. I am under the impression that 
these tests don't get run all the time so we might want to run that suite.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-14 Thread Joe Jevnik

Joe Jevnik added the comment:

Has the test passed or is it still hung? Also, I guess this means that the 
tuple is over-allocating, maybe I need to revise the memory assumptions to 
account for the growth algorithm.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-14 Thread Joe Jevnik

Joe Jevnik added the comment:

I think that the idea is not to get as close to the limit as possible but just 
to hard cap the memory usage of the test suite so that it doesn't get 
oom-killed. twouters, does this sound correct? Also, I think that this means 
that the new decorator is reporting a proper size hint.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-14 Thread Joe Jevnik

Joe Jevnik added the comment:

I am updating the patch to include an entry in Misc/NEWS.

--
Added file: http://bugs.python.org/file38994/namedtuple-indexer-with-news.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22631] Feature Request CAN_RAW_FD_FRAMES

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

I am not going to be able to write docs on this because I am not totally sure 
how you would use this; There is an entry in the docs for CAN_* so that might 
be enough. I have a line now saying that this particular flag is available as 
of 3.5 though.

--
Added file: http://bugs.python.org/file38946/CAN_RAW_FD_FRAMES-update.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22631
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

I am now computing the size based on the system's int and pointer size.

--
keywords: +patch
nosy: +ll
Added file: http://bugs.python.org/file38945/bigmem.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22631] Feature Request CAN_RAW_FD_FRAMES

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

I tried to provide a summary of the Linux page on the topic and then said that 
it was fully documented there.

--
Added file: http://bugs.python.org/file38957/CAN_RAW_FD_FRAMES-doc.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22631
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

I have updated the test to make another test that does not have a __len__.

--
Added file: http://bugs.python.org/file38952/bigmem-update.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23926] skipitem() in getargs.c still supports 'w' and 'w#', and shouldn't

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

here is a patch and test case.
I added an entry to Misc/NEWS. I tried to follow the format; however, let me 
know if I have done this incorrectly.

--
keywords: +patch
nosy: +ll
Added file: http://bugs.python.org/file38972/skipitems.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23926
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22631] Feature Request CAN_RAW_FD_FRAMES

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

So I think the patch could be applied to 3.4; I will look into moving the other 
back sometime this week.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22631
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5438] test_bigmem.test_from_2G_generator uses more memory than expected

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

Steve, we talked about the possibility of having you run some of these tests on 
a machine with a large amount of ram; I am adding you to the nosy list.

--
nosy: +steve.dower

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5438
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22631] Feature Request CAN_RAW_FD_FRAMES

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

I have added a patch to update this with conditional compilation.

--
keywords: +patch
nosy: +ll
Added file: http://bugs.python.org/file38934/CAN_RAW_FD_FRAMES.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22631
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22631] Feature Request CAN_RAW_FD_FRAMES

2015-04-13 Thread Joe Jevnik

Joe Jevnik added the comment:

To show where I got my sources: 
https://github.com/torvalds/linux/commit/e2d265d3b587f5f6f8febc0222aace93302ff0be

There does not appear to be any new structures needed other than supporting the 
constant.

--
nosy: +larry

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22631
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-12 Thread Joe Jevnik

Joe Jevnik added the comment:

sorry, I meant pypy

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue23910] C implementation of namedtuple (WIP)

2015-04-12 Thread Joe Jevnik

Joe Jevnik added the comment:

# Original version / new python implementation
./python -m timeit -s from collections import namedtuple;a = namedtuple('a', 
'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.07 usec per loop


# C implementation
./python -m timeit -s from collections import namedtuple;a = namedtuple('a', 
'a b c')(1, 2, 3) a.a
1000 loops, best of 3: 0.028 usec per loop


The fallback is the same implementation that is currently used so this should 
have no affect on pypi.

--
Added file: http://bugs.python.org/file38905/namedtuple-indexer-update.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23910
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   >