Question #631423 on Duplicity changed: https://answers.launchpad.net/duplicity/+question/631423
Howard Kaye posted a new comment: I think it just moved the error somewhere else. On May 17, 2017 4:09 AM, "edso" <[email protected]> wrote: > Your question #631423 on Duplicity changed: > https://answers.launchpad.net/duplicity/+question/631423 > > edso posted a new comment: > nice catch Howie! > > wonder why this worked with os.tmpfile() but not with > tempfile.TemporaryFile() before though.. ede/duply.net > > On 16.05.2017 20:33, Howard Kaye wrote: > > Question #631423 on Duplicity changed: > > https://answers.launchpad.net/duplicity/+question/631423 > > > > Status: Answered => Solved > > > > Howard Kaye confirmed that the question is solved: > > OK, I found and fixed the root cause of this error: > > > > Duplicity was failing in the glue code (in C), between Python and > librsync. > > It did an fdopen() on the file descriptor from a Python File object, and > then never > > closed the File * which was returned. Even though the Python File > object (and the > > file descriptor shared by the File * and the Python File) got closed > when it was > > deallocated, the File *'s were being leaked, and eventually the stdio > library ran > > out of File *'s (even though the number of open files was not that > large). > > > > The fix is to dup the file descriptor, and then close the file in the > deallocator > > routine in the glue code. Duping the file lets the C code and the > Python code > > each close the file, when they are done with it. > > > > *** _librsyncmodule.c.orig 2017-01-06 09:23:21.000000000 -0500 > > --- _librsyncmodule.c 2017-05-16 11:12:58.000000000 -0400 > > *************** > > *** 23,28 **** > > --- 23,29 ---- > > * ----------------------------------------------------------------------- > */ > > > > #include <Python.h> > > + #include <errno.h> > > #include <librsync.h> > > #define RS_JOB_BLOCKSIZE 65536 > > > > *************** > > *** 287,292 **** > > --- 288,294 ---- > > PyObject_HEAD > > rs_job_t *patch_job; > > PyObject *basis_file; > > + FILE *cfile; > > } _librsync_PatchMakerObject; > > > > /* Call with the basis file */ > > *************** > > *** 296,302 **** > > _librsync_PatchMakerObject* pm; > > PyObject *python_file; > > int fd; > > - FILE *cfile; > > > > if (!PyArg_ParseTuple(args, "O:new_patchmaker", &python_file)) > > return NULL; > > --- 298,303 ---- > > *************** > > *** 305,318 **** > > PyErr_SetString(PyExc_TypeError, "Need true file object"); > > return NULL; > > } > > Py_INCREF(python_file); > > > > pm = PyObject_New(_librsync_PatchMakerObject, > &_librsync_PatchMakerType); > > if (pm == NULL) return NULL; > > > > pm->basis_file = python_file; > > ! cfile = fdopen(fd, "rb"); > > ! pm->patch_job = rs_patch_begin(rs_file_copy_cb, cfile); > > > > return (PyObject*)pm; > > } > > --- 306,327 ---- > > PyErr_SetString(PyExc_TypeError, "Need true file object"); > > return NULL; > > } > > + /* get our own private copy of the file, so we can close it later. */ > > + fd = dup(fd); > > + if (fd == -1) { > > + char buf[256]; > > + strerror_r(errno, buf, sizeof(buf)); > > + PyErr_SetString(PyExc_TypeError, buf); > > + return NULL; > > + } > > Py_INCREF(python_file); > > > > pm = PyObject_New(_librsync_PatchMakerObject, > &_librsync_PatchMakerType); > > if (pm == NULL) return NULL; > > > > pm->basis_file = python_file; > > ! pm->cfile = fdopen(fd, "rb"); > > ! pm->patch_job = rs_patch_begin(rs_file_copy_cb, pm->cfile); > > > > return (PyObject*)pm; > > } > > *************** > > *** 323,328 **** > > --- 332,340 ---- > > _librsync_PatchMakerObject *pm = (_librsync_PatchMakerObject *)self; > > Py_DECREF(pm->basis_file); > > rs_job_free(pm->patch_job); > > + if (pm->cfile) { > > + fclose(pm->cfile); > > + } > > PyObject_Del(self); > > } > > > > -- > You received this question notification because you asked the question. > -- You received this question notification because your team duplicity-team is an answer contact for Duplicity. _______________________________________________ Mailing list: https://launchpad.net/~duplicity-team Post to : [email protected] Unsubscribe : https://launchpad.net/~duplicity-team More help : https://help.launchpad.net/ListHelp
