Question #631423 on Duplicity changed: https://answers.launchpad.net/duplicity/+question/631423
edso posted a new comment: Howie, Ken, would you agree then, that the whole os.tmpfile() exception http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8-series/view/head:/duplicity/patchdir.py#L498 " if sys.platform.startswith(('cygwin', 'windows', 'darwin')): tempfp = os.tmpfile() else: " can be removed after the librsync adapter is patched?.. ede On 17.05.2017 12:27, Howard Kaye wrote: > Question #631423 on Duplicity changed: > https://answers.launchpad.net/duplicity/+question/631423 > > Howard Kaye posted a new comment: > I think it just moved the error somewhere else. > > On May 17, 2017 4:09 AM, "edso" <question631...@answers.launchpad.net> > wrote: > >> Your question #631423 on Duplicity changed: >> https://answers.launchpad.net/duplicity/+question/631423 >> >> edso posted a new comment: >> nice catch Howie! >> >> wonder why this worked with os.tmpfile() but not with >> tempfile.TemporaryFile() before though.. ede/duply.net >> >> On 16.05.2017 20:33, Howard Kaye wrote: >>> Question #631423 on Duplicity changed: >>> https://answers.launchpad.net/duplicity/+question/631423 >>> >>> Status: Answered => Solved >>> >>> Howard Kaye confirmed that the question is solved: >>> OK, I found and fixed the root cause of this error: >>> >>> Duplicity was failing in the glue code (in C), between Python and >> librsync. >>> It did an fdopen() on the file descriptor from a Python File object, and >> then never >>> closed the File * which was returned. Even though the Python File >> object (and the >>> file descriptor shared by the File * and the Python File) got closed >> when it was >>> deallocated, the File *'s were being leaked, and eventually the stdio >> library ran >>> out of File *'s (even though the number of open files was not that >> large). >>> >>> The fix is to dup the file descriptor, and then close the file in the >> deallocator >>> routine in the glue code. Duping the file lets the C code and the >> Python code >>> each close the file, when they are done with it. >>> >>> *** _librsyncmodule.c.orig 2017-01-06 09:23:21.000000000 -0500 >>> --- _librsyncmodule.c 2017-05-16 11:12:58.000000000 -0400 >>> *************** >>> *** 23,28 **** >>> --- 23,29 ---- >>> * ----------------------------------------------------------------------- >> */ >>> >>> #include <Python.h> >>> + #include <errno.h> >>> #include <librsync.h> >>> #define RS_JOB_BLOCKSIZE 65536 >>> >>> *************** >>> *** 287,292 **** >>> --- 288,294 ---- >>> PyObject_HEAD >>> rs_job_t *patch_job; >>> PyObject *basis_file; >>> + FILE *cfile; >>> } _librsync_PatchMakerObject; >>> >>> /* Call with the basis file */ >>> *************** >>> *** 296,302 **** >>> _librsync_PatchMakerObject* pm; >>> PyObject *python_file; >>> int fd; >>> - FILE *cfile; >>> >>> if (!PyArg_ParseTuple(args, "O:new_patchmaker", &python_file)) >>> return NULL; >>> --- 298,303 ---- >>> *************** >>> *** 305,318 **** >>> PyErr_SetString(PyExc_TypeError, "Need true file object"); >>> return NULL; >>> } >>> Py_INCREF(python_file); >>> >>> pm = PyObject_New(_librsync_PatchMakerObject, >> &_librsync_PatchMakerType); >>> if (pm == NULL) return NULL; >>> >>> pm->basis_file = python_file; >>> ! cfile = fdopen(fd, "rb"); >>> ! pm->patch_job = rs_patch_begin(rs_file_copy_cb, cfile); >>> >>> return (PyObject*)pm; >>> } >>> --- 306,327 ---- >>> PyErr_SetString(PyExc_TypeError, "Need true file object"); >>> return NULL; >>> } >>> + /* get our own private copy of the file, so we can close it later. */ >>> + fd = dup(fd); >>> + if (fd == -1) { >>> + char buf[256]; >>> + strerror_r(errno, buf, sizeof(buf)); >>> + PyErr_SetString(PyExc_TypeError, buf); >>> + return NULL; >>> + } >>> Py_INCREF(python_file); >>> >>> pm = PyObject_New(_librsync_PatchMakerObject, >> &_librsync_PatchMakerType); >>> if (pm == NULL) return NULL; >>> >>> pm->basis_file = python_file; >>> ! pm->cfile = fdopen(fd, "rb"); >>> ! pm->patch_job = rs_patch_begin(rs_file_copy_cb, pm->cfile); >>> >>> return (PyObject*)pm; >>> } >>> *************** >>> *** 323,328 **** >>> --- 332,340 ---- >>> _librsync_PatchMakerObject *pm = (_librsync_PatchMakerObject *)self; >>> Py_DECREF(pm->basis_file); >>> rs_job_free(pm->patch_job); >>> + if (pm->cfile) { >>> + fclose(pm->cfile); >>> + } >>> PyObject_Del(self); >>> } >>> >> >> -- >> You received this question notification because you asked the question. >> > -- You received this question notification because your team duplicity-team is an answer contact for Duplicity. _______________________________________________ Mailing list: https://launchpad.net/~duplicity-team Post to : duplicity-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~duplicity-team More help : https://help.launchpad.net/ListHelp