Vihang Karajgaonkar has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/17110 )
Change subject: IMPALA-10450: Catalogd crashes due to exception in ThriftDebugString ...................................................................... IMPALA-10450: Catalogd crashes due to exception in ThriftDebugString This patch adds a wrapper around ThriftDebugString method provided in the Thrift library. The thrift's method can throw exceptions like (bad_alloc or TProtocolException) when the object cannot be serialized into a string representation. This exception is not caught on the catalogd side and it crashes the catalogd. The error was specifically seen in the catalogd's debug UI which provides a way to display a Table object. An exception thrown when rendering the table on the UI would have crashed the catalogd before the patch. In order to simulate this crash a new debug action called EXCEPTION was added. A new custom cluster test was added which simulates a exception thrown in this method and makes sure that fetching the table from catalogd's debug UI does not crash the catalogd. Tests: 1. Added a new custom cluster test which reproduces the crash. 2. Created a large table which has ~270K partitions and reduced the memory of the catalogd to 16GB. This configuration throws bad_alloc exception in the ThriftDebugString method and crashes the catalogd. After the patch the crash is averted and we see a error message on the debug UI instead. I also looped around the catalog web UI call for more than an hour to see if there are any other stability issues. I could not see any problems. Change-Id: I42cee6186a3d5bacc1117bae5961ac60ac9f7a66 --- M be/src/catalog/catalog-server.cc M be/src/util/debug-util.cc M be/src/util/debug-util.h A be/src/util/thrift-debug-util.h A tests/custom_cluster/test_thrift_debug_string_exception.py 5 files changed, 121 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/17110/9 -- To view, visit http://gerrit.cloudera.org:8080/17110 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I42cee6186a3d5bacc1117bae5961ac60ac9f7a66 Gerrit-Change-Number: 17110 Gerrit-PatchSet: 9 Gerrit-Owner: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
