Re: AW: [firebird-python] FDB v1.4.2
Hi, Fixed in 1.4.3, see PyPI. Thanks for pointing this out. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 11:42 Dominik Psenner dominik.psen...@topcontrol.it [firebird-python] napsal(a): Hi Pavel, great to hear that you have got this release out of the door so quickly. However, I’m afraid that http://tracker.firebirdsql.org/browse/PYFB-43 is not yet resolved. As posted in the issue I was just now able to reproduce the memory leak with 1.4.2. Thus, people, beware from invoking event_conduit() in a loop! Are there any plans for fixing the memory leak? Best regards, Dominik Von: firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 10:36 An: firebird-python@yahoogroups.com Betreff: [firebird-python] FDB v1.4.2 Hi, FDB Python driver for Firebird v1.4.2 is available for download . This mostly bugfix version introduces a backward incompatible change to events API, so read the Release Notes before deployment. https://pypi.python.org/pypi/fdb best regards Pavel Cisar IBPhoenix
AW: AW: [firebird-python] FDB v1.4.2 [1 Attachment]
Ah, you were faster than I could write. Sorry for me being so persistent. Unfortunately I am unable to post a file in the bug tracker now that the issue is resolved, therefore I write you here. The good thing is, 1.4.3 improved the situation. One cyclic reference has gone away but there’s still one cyclic reference left to be broken. The attached objgraph shows that the one left might be the callback function which is stored in the EventBlock upon creation. You could break that by cleaning up the callback attribute on close(). Doing that should resolve this issue for good. Von: firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 12:06 An: firebird-python@yahoogroups.com Betreff: AW: AW: [firebird-python] FDB v1.4.2 Awesome! Thanks for the fast responses, you’re doing a good job. ☺ If you feel like sending me a patch that I can apply before you push 1.4.3 out of the door, I’ll gladly run my test scripts on a patched 1.4.2. Cheers Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 11:50 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com Betreff: Re: AW: [firebird-python] FDB v1.4.2 Hi, Fixed in 1.4.3, see PyPI. Thanks for pointing this out. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 11:42 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): Hi Pavel, great to hear that you have got this release out of the door so quickly. However, I’m afraid that http://tracker.firebirdsql.org/browse/PYFB-43 is not yet resolved. As posted in the issue I was just now able to reproduce the memory leak with 1.4.2. Thus, people, beware from invoking event_conduit() in a loop! Are there any plans for fixing the memory leak? Best regards, Dominik Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 10:36 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com Betreff: [firebird-python] FDB v1.4.2 Hi, FDB Python driver for Firebird v1.4.2 is available for download . This mostly bugfix version introduces a backward incompatible change to events API, so read the Release Notes before deployment. https://pypi.python.org/pypi/fdb best regards Pavel Cisar IBPhoenix
Re: AW: AW: [firebird-python] FDB v1.4.2
Well, 1.4.3 is already out. Dne 14.11.2014 v 12:06 Dominik Psenner dominik.psen...@topcontrol.it [firebird-python] napsal(a): Awesome! Thanks for the fast responses, you’re doing a good job. ☺ If you feel like sending me a patch that I can apply before you push 1.4.3 out of the door, I’ll gladly run my test scripts on a patched 1.4.2. Cheers Von: firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 11:50 An: firebird-python@yahoogroups.com Betreff: Re: AW: [firebird-python] FDB v1.4.2 Hi, Fixed in 1.4.3, see PyPI. Thanks for pointing this out. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 11:42 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): Hi Pavel, great to hear that you have got this release out of the door so quickly. However, I’m afraid that http://tracker.firebirdsql.org/browse/PYFB-43 is not yet resolved. As posted in the issue I was just now able to reproduce the memory leak with 1.4.2. Thus, people, beware from invoking event_conduit() in a loop! Are there any plans for fixing the memory leak? Best regards, Dominik Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 10:36 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com Betreff: [firebird-python] FDB v1.4.2 Hi, FDB Python driver for Firebird v1.4.2 is available for download . This mostly bugfix version introduces a backward incompatible change to events API, so read the Release Notes before deployment. https://pypi.python.org/pypi/fdb best regards Pavel Cisar IBPhoenix
AW: AW: AW: [firebird-python] FDB v1.4.2
I refreshed the package and it looks like the memory leak is almost gone. I can observe that the number of EventBlock’s left in memory are equal to the count of errors raised with the invoke of begin() (namely one of: unknown ISC error 0, error reading from connection, error writing to connection). So there’s still a troublesome execution path left and it took some time until I had wrapped my head around it. According to the stack trace the exception is thrown in the constructor of EventBlock: File dbEvents.py, line 103, in run event_listener.begin() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1844, in begin self.__event_blocks.append(EventBlock(self.__queue, self._db_handle, block_events)) File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1732, in __init__ self.__wait_for_events() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1741, in __wait_for_events Error while waiting for events:) DatabaseError: ('Error while waiting for events:\n- SQLCODE: -902\n- Error reading data from the connection.', -902, 335544726) That would be right here where the EventBlock should be appended to the list of event blocks: for block_events in self.__blocks: self.__event_blocks.append(EventBlock(self.__queue, self._db_handle, block_events)) But like this an initialized and broken EventBlock is never appended to the __event_blocks list and therefore EventBlock.close() is never invoked. ☹ Von: firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 13:38 An: firebird-python@yahoogroups.com Betreff: Re: AW: AW: [firebird-python] FDB v1.4.2 Hi, Fixed. This time I've just refreshed the 1.4.3 package, as I don't want to make another point release in single day. If anyone already got unpatched 1.4.3 (unlikely that anyone beside you get it so quickly), then it will replaced by next version 1.5 (full Firebird 3.0 support) scheduled for release before end of year. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 12:24 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): Ah, you were faster than I could write. Sorry for me being so persistent. Unfortunately I am unable to post a file in the bug tracker now that the issue is resolved, therefore I write you here. The good thing is, 1.4.3 improved the situation. One cyclic reference has gone away but there’s still one cyclic reference left to be broken. The attached objgraph shows that the one left might be the callback function which is stored in the EventBlock upon creation. You could break that by cleaning up the callback attribute on close(). Doing that should resolve this issue for good. Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 12:06 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com Betreff: AW: AW: [firebird-python] FDB v1.4.2 Awesome! Thanks for the fast responses, you’re doing a good job. ☺ If you feel like sending me a patch that I can apply before you push 1.4.3 out of the door, I’ll gladly run my test scripts on a patched 1.4.2. Cheers Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com%3cmailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 11:50 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com%3cmailto:firebird-python@yahoogroups.com Betreff: Re: AW: [firebird-python] FDB v1.4.2 Hi, Fixed in 1.4.3, see PyPI. Thanks for pointing this out. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 11:42 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it%3cmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): Hi Pavel, great to hear that you have got this release out of the door so quickly. However, I’m afraid that http://tracker.firebirdsql.org/browse/PYFB-43 is not yet resolved. As posted in the issue I was just now able to reproduce the memory leak with 1.4.2. Thus, people, beware from invoking event_conduit() in a loop! Are there any plans for fixing the memory leak? Best regards, Dominik Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com%3cmailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 10:36 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com%3cmailto:firebird-python@yahoogroups.com Betreff: [firebird-python] FDB v1.4.2 Hi, FDB Python driver
Re: AW: AW: AW: [firebird-python] FDB v1.4.2
Hi, Can you try to add a call to self.close() before exception is raised in __wait_for_events ? Does it helps? if db_api_error(self._isc_status): self.close() raise exception_from_status(DatabaseError, self._isc_status, Error while waiting for events:) best regards Pavel Dne 14.11.2014 v 14:08 Dominik Psenner dominik.psen...@topcontrol.it [firebird-python] napsal(a): I refreshed the package and it looks like the memory leak is almost gone. I can observe that the number of EventBlock’s left in memory are equal to the count of errors raised with the invoke of begin() (namely one of: unknown ISC error 0, error reading from connection, error writing to connection). So there’s still a troublesome execution path left and it took some time until I had wrapped my head around it. According to the stack trace the exception is thrown in the constructor of EventBlock: File dbEvents.py, line 103, in run event_listener.begin() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1844, in begin self.__event_blocks.append(EventBlock(self.__queue, self._db_handle, block_events)) File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1732, in __init__ self.__wait_for_events() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1741, in __wait_for_events Error while waiting for events:) DatabaseError: ('Error while waiting for events:\n- SQLCODE: -902\n- Error reading data from the connection.', -902, 335544726) That would be right here where the EventBlock should be appended to the list of event blocks: for block_events in self.__blocks: self.__event_blocks.append(EventBlock(self.__queue, self._db_handle, block_events)) But like this an initialized and broken EventBlock is never appended to the __event_blocks list and therefore EventBlock.close() is never invoked. ☹ Von: firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 13:38 An: firebird-python@yahoogroups.com Betreff: Re: AW: AW: [firebird-python] FDB v1.4.2 Hi, Fixed. This time I've just refreshed the 1.4.3 package, as I don't want to make another point release in single day. If anyone already got unpatched 1.4.3 (unlikely that anyone beside you get it so quickly), then it will replaced by next version 1.5 (full Firebird 3.0 support) scheduled for release before end of year. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 12:24 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): Ah, you were faster than I could write. Sorry for me being so persistent. Unfortunately I am unable to post a file in the bug tracker now that the issue is resolved, therefore I write you here. The good thing is, 1.4.3 improved the situation. One cyclic reference has gone away but there’s still one cyclic reference left to be broken. The attached objgraph shows that the one left might be the callback function which is stored in the EventBlock upon creation. You could break that by cleaning up the callback attribute on close(). Doing that should resolve this issue for good. Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 12:06 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com Betreff: AW: AW: [firebird-python] FDB v1.4.2 Awesome! Thanks for the fast responses, you’re doing a good job. ☺ If you feel like sending me a patch that I can apply before you push 1.4.3 out of the door, I’ll gladly run my test scripts on a patched 1.4.2. Cheers Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com%3cmailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 11:50 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com%3cmailto:firebird-python@yahoogroups.com Betreff: Re: AW: [firebird-python] FDB v1.4.2 Hi, Fixed in 1.4.3, see PyPI. Thanks for pointing this out. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 11:42 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it%3cmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): Hi Pavel, great to hear that you have got this release out of the door so quickly. However, I’m afraid that http://tracker.firebirdsql.org/browse/PYFB-43 is not yet resolved. As posted in the issue I was just now able to reproduce the memory leak with 1.4.2. Thus, people, beware from invoking event_conduit() in a loop! Are there any plans for fixing the memory leak? Best regards, Dominik Von:
AW: AW: AW: AW: [firebird-python] FDB v1.4.2
*hm* .. not really.. then an AttributeError pops up: 2014-11-14 15:15:08.498000 50/51 (1|50:51|3|0) event_conduit(['TC_IMPORT_QUEUE_UPDATED'])Exception in thread Thread-6: Traceback (most recent call last): File C:\Python27\lib\threading.py, line 810, in __bootstrap_inner self.run() File C:\Python27\lib\threading.py, line 763, in run self.__target(*self.__args, **self.__kwargs) File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1831, in event_process events = data.count_and_reregister() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1751, in count_and_reregister self.__wait_for_events() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1742, in __wait_for_events Error while waiting for events:) DatabaseError: ('Error while waiting for events:\n- SQLCODE: 0\n- unknown ISC error 0', 0, 0) 2014-11-14 15:15:08.608000 50/52 (2|25:26|4|1) Traceback (most recent call last): File dbEvents.py, line 103, in run event_listener.begin() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1845, in begin self.__event_blocks.append(EventBlock(self.__queue, self._db_handle, block_events)) File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1732, in __init__ self.__wait_for_events() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1740, in __wait_for_events self.close() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1763, in close del self.__callback AttributeError: _EventBlock__callback Von: firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 15:03 An: firebird-python@yahoogroups.com Betreff: Re: AW: AW: AW: [firebird-python] FDB v1.4.2 Hi, Can you try to add a call to self.close() before exception is raised in __wait_for_events ? Does it helps? if db_api_error(self._isc_status): self.close() raise exception_from_status(DatabaseError, self._isc_status, Error while waiting for events:) best regards Pavel Dne 14.11.2014 v 14:08 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): I refreshed the package and it looks like the memory leak is almost gone. I can observe that the number of EventBlock’s left in memory are equal to the count of errors raised with the invoke of begin() (namely one of: unknown ISC error 0, error reading from connection, error writing to connection). So there’s still a troublesome execution path left and it took some time until I had wrapped my head around it. According to the stack trace the exception is thrown in the constructor of EventBlock: File dbEvents.py, line 103, in run event_listener.begin() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1844, in begin self.__event_blocks.append(EventBlock(self.__queue, self._db_handle, block_events)) File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1732, in __init__ self.__wait_for_events() File C:\Python27\lib\site-packages\fdb-1.4.3-py2.7.egg\fdb\fbcore.py, line 1741, in __wait_for_events Error while waiting for events:) DatabaseError: ('Error while waiting for events:\n- SQLCODE: -902\n- Error reading data from the connection.', -902, 335544726) That would be right here where the EventBlock should be appended to the list of event blocks: for block_events in self.__blocks: self.__event_blocks.append(EventBlock(self.__queue, self._db_handle, block_events)) But like this an initialized and broken EventBlock is never appended to the __event_blocks list and therefore EventBlock.close() is never invoked. ☹ Von: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com [mailto:firebird-python@yahoogroups.com] Gesendet: Freitag, 14. November 2014 13:38 An: firebird-python@yahoogroups.commailto:firebird-python@yahoogroups.com Betreff: Re: AW: AW: [firebird-python] FDB v1.4.2 Hi, Fixed. This time I've just refreshed the 1.4.3 package, as I don't want to make another point release in single day. If anyone already got unpatched 1.4.3 (unlikely that anyone beside you get it so quickly), then it will replaced by next version 1.5 (full Firebird 3.0 support) scheduled for release before end of year. best regards Pavel Cisar IBPhoenix Dne 14.11.2014 v 12:24 Dominik Psenner dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.itmailto:dominik.psen...@topcontrol.it%3cmailto:dominik.psen...@topcontrol.it [firebird-python] napsal(a): Ah, you were faster than I could write. Sorry for me being so persistent. Unfortunately I am unable to post a file in the bug tracker now that the issue is resolved, therefore I write you here. The good thing is, 1.4.3 improved the situation. One cyclic reference has gone away but there’s still one cyclic reference left to