Question #631423 on Duplicity changed: https://answers.launchpad.net/duplicity/+question/631423
Howard Kaye posted a new comment: Yes, it is not needed at all. On Wed, May 17, 2017 at 6:47 AM, edso <[email protected]> wrote: > Your question #631423 on Duplicity changed: > https://answers.launchpad.net/duplicity/+question/631423 > > edso posted a new comment: > Howie, Ken, > > would you agree then, that the whole os.tmpfile() exception > > http://bazaar.launchpad.net/~duplicity-team/duplicity/0.8- > series/view/head:/duplicity/patchdir.py#L498 > " > if sys.platform.startswith(('cygwin', 'windows', 'darwin')): > tempfp = os.tmpfile() > else: > " > > can be removed after the librsync adapter is patched?.. ede > > On 17.05.2017 12:27, Howard Kaye wrote: > > Question #631423 on Duplicity changed: > > https://answers.launchpad.net/duplicity/+question/631423 > > > > Howard Kaye posted a new comment: > > I think it just moved the error somewhere else. > > > > On May 17, 2017 4:09 AM, "edso" <[email protected]> > > wrote: > > > >> Your question #631423 on Duplicity changed: > >> https://answers.launchpad.net/duplicity/+question/631423 > >> > >> edso posted a new comment: > >> nice catch Howie! > >> > >> wonder why this worked with os.tmpfile() but not with > >> tempfile.TemporaryFile() before though.. ede/duply.net > >> > >> On 16.05.2017 20:33, Howard Kaye wrote: > >>> Question #631423 on Duplicity changed: > >>> https://answers.launchpad.net/duplicity/+question/631423 > >>> > >>> Status: Answered => Solved > >>> > >>> Howard Kaye confirmed that the question is solved: > >>> OK, I found and fixed the root cause of this error: > >>> > >>> Duplicity was failing in the glue code (in C), between Python and > >> librsync. > >>> It did an fdopen() on the file descriptor from a Python File object, > and > >> then never > >>> closed the File * which was returned. Even though the Python File > >> object (and the > >>> file descriptor shared by the File * and the Python File) got closed > >> when it was > >>> deallocated, the File *'s were being leaked, and eventually the stdio > >> library ran > >>> out of File *'s (even though the number of open files was not that > >> large). > >>> > >>> The fix is to dup the file descriptor, and then close the file in the > >> deallocator > >>> routine in the glue code. Duping the file lets the C code and the > >> Python code > >>> each close the file, when they are done with it. > >>> > >>> *** _librsyncmodule.c.orig 2017-01-06 09:23:21.000000000 -0500 > >>> --- _librsyncmodule.c 2017-05-16 11:12:58.000000000 -0400 > >>> *************** > >>> *** 23,28 **** > >>> --- 23,29 ---- > >>> * ------------------------------------------------------------ > ----------- > >> */ > >>> > >>> #include <Python.h> > >>> + #include <errno.h> > >>> #include <librsync.h> > >>> #define RS_JOB_BLOCKSIZE 65536 > >>> > >>> *************** > >>> *** 287,292 **** > >>> --- 288,294 ---- > >>> PyObject_HEAD > >>> rs_job_t *patch_job; > >>> PyObject *basis_file; > >>> + FILE *cfile; > >>> } _librsync_PatchMakerObject; > >>> > >>> /* Call with the basis file */ > >>> *************** > >>> *** 296,302 **** > >>> _librsync_PatchMakerObject* pm; > >>> PyObject *python_file; > >>> int fd; > >>> - FILE *cfile; > >>> > >>> if (!PyArg_ParseTuple(args, "O:new_patchmaker", &python_file)) > >>> return NULL; > >>> --- 298,303 ---- > >>> *************** > >>> *** 305,318 **** > >>> PyErr_SetString(PyExc_TypeError, "Need true file object"); > >>> return NULL; > >>> } > >>> Py_INCREF(python_file); > >>> > >>> pm = PyObject_New(_librsync_PatchMakerObject, > >> &_librsync_PatchMakerType); > >>> if (pm == NULL) return NULL; > >>> > >>> pm->basis_file = python_file; > >>> ! cfile = fdopen(fd, "rb"); > >>> ! pm->patch_job = rs_patch_begin(rs_file_copy_cb, cfile); > >>> > >>> return (PyObject*)pm; > >>> } > >>> --- 306,327 ---- > >>> PyErr_SetString(PyExc_TypeError, "Need true file object"); > >>> return NULL; > >>> } > >>> + /* get our own private copy of the file, so we can close it later. > */ > >>> + fd = dup(fd); > >>> + if (fd == -1) { > >>> + char buf[256]; > >>> + strerror_r(errno, buf, sizeof(buf)); > >>> + PyErr_SetString(PyExc_TypeError, buf); > >>> + return NULL; > >>> + } > >>> Py_INCREF(python_file); > >>> > >>> pm = PyObject_New(_librsync_PatchMakerObject, > >> &_librsync_PatchMakerType); > >>> if (pm == NULL) return NULL; > >>> > >>> pm->basis_file = python_file; > >>> ! pm->cfile = fdopen(fd, "rb"); > >>> ! pm->patch_job = rs_patch_begin(rs_file_copy_cb, pm->cfile); > >>> > >>> return (PyObject*)pm; > >>> } > >>> *************** > >>> *** 323,328 **** > >>> --- 332,340 ---- > >>> _librsync_PatchMakerObject *pm = (_librsync_PatchMakerObject > *)self; > >>> Py_DECREF(pm->basis_file); > >>> rs_job_free(pm->patch_job); > >>> + if (pm->cfile) { > >>> + fclose(pm->cfile); > >>> + } > >>> PyObject_Del(self); > >>> } > >>> > >> > >> -- > >> You received this question notification because you asked the question. > >> > > > > -- > You received this question notification because you asked the question. > -- You received this question notification because your team duplicity-team is an answer contact for Duplicity. _______________________________________________ Mailing list: https://launchpad.net/~duplicity-team Post to : [email protected] Unsubscribe : https://launchpad.net/~duplicity-team More help : https://help.launchpad.net/ListHelp
