Karl Nelson <nelso...@llnl.gov> added the comment:
Thanks, I will see if I can get additional diagnostics today. Otherwise I will
have to start recompiling Python with diagnostic hooks. I have never had to
rebuild python on windows so it may take a while to figure out how to make
progress.
The monkey patch is less scary than it looks. Since I only need these slots
of specific basic types that I designate (not user selected), the extra types
all extend without altering the underlaying type, and the class layout is
controlled by a meta class so that only Java can create these type objects, it
is reasonably robust. However, yes changes in Python can break it. It is
basically an attempt to create a mixin for concrete classes without the benefit
of a dictoffset type slot. It took about a month do develop something that
safely added the slot without overrunning, leaking or creating other memory
issues (and played nice when Python added its own dict slot).
Unfortunately, as I do many 100s on JSlot tests during method resolution using
a dict based approach was unacceptably slow. I would love if there was a
formal method to create multiple inheritance mixins for Python basic types, but
that is another issue.
```
/**
* Allocate a new Python object with a slot for Java.
*
* We need extra space to store our values, but because there
* is no way to do so without disturbing the object layout.
* Fortunately, Python already handles this for dict and weakref.
* Python aligns the ends of the structure and increases the
* base type size to add additional slots to a standard object.
*
* We will use the same trick to add an additional slot for Java
* after the end of the object outside of where Python is looking.
* As the memory is aligned this is safe to do. We will use
* the alloc and finalize slot to recognize which objects have this
* extra slot appended.
*/
PyObject* PyJPValue_alloc(PyTypeObject* type, Py_ssize_t nitems )
{
// Modification from Python to add size elements
const size_t size = _PyObject_VAR_SIZE(type, nitems + 1) + sizeof
(JPValue);
PyObject *obj = (PyType_IS_GC(type)) ? _PyObject_GC_Malloc(size)
: (PyObject *) PyObject_MALLOC(size);
if (obj == NULL)
return PyErr_NoMemory(); // GCOVR_EXCL_LINE
memset(obj, 0, size);
Py_ssize_t refcnt = ((PyObject*) type)->ob_refcnt;
if (type->tp_itemsize == 0)
PyObject_Init(obj, type);
else
PyObject_InitVar((PyVarObject *) obj, type, nitems);
// This line is required to deal with Python bug (GH-11661)
// Some versions of Python fail to increment the reference counter of
// heap types properly.
if (refcnt == ((PyObject*) type)->ob_refcnt)
Py_INCREF(type); // GCOVR_EXCL_LINE
if (PyType_IS_GC(type))
{
PyObject_GC_Track(obj);
}
return obj;
}
bool PyJPValue_hasJavaSlot(PyTypeObject* type)
{
if (type == NULL
|| type->tp_alloc != (allocfunc) PyJPValue_alloc
|| type->tp_finalize != (destructor) PyJPValue_finalize)
return false; // GCOVR_EXCL_LINE
return true;
}
// Check for a Java slot (must work on all object types)
Py_ssize_t PyJPValue_getJavaSlotOffset(PyObject* self)
{
PyTypeObject *type = Py_TYPE(self);
if (type == NULL
|| type->tp_alloc != (allocfunc) PyJPValue_alloc
|| type->tp_finalize != (destructor) PyJPValue_finalize)
return 0;
Py_ssize_t offset;
Py_ssize_t sz = Py_SIZE(self);
// I have no clue what negative sizes mean
if (sz < 0)
sz = -sz;
if (type->tp_itemsize == 0)
offset = _PyObject_VAR_SIZE(type, 1);
else
offset = _PyObject_VAR_SIZE(type, sz + 1);
return offset;
}
JPValue* PyJPValue_getJavaSlot(PyObject* self)
{
Py_ssize_t offset = PyJPValue_getJavaSlotOffset(self);
if (offset == 0)
return NULL;
JPValue* value = (JPValue*) (((char*) self) + offset);
if (value->getClass() == NULL)
return NULL;
return value;
}
void PyJPValue_free(void* obj)
{
// Normally finalize is not run on simple classes.
PyTypeObject *type = Py_TYPE(obj);
if (type->tp_finalize != NULL)
type->tp_finalize((PyObject*) obj);
if (type->tp_flags & Py_TPFLAGS_HAVE_GC)
PyObject_GC_Del(obj);
else
PyObject_Free(obj); // GCOVR_EXCL_LINE
}
void PyJPValue_finalize(void* obj)
{
JPValue* value = PyJPValue_getJavaSlot((PyObject*) obj);
... destroy the objects hidden slot if it exists
}
```
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue42529>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
