Yida Wu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16638
Change subject: IMPALA-10102 Fix Impalad crashses when writting a parquet file with large rows. ...................................................................... IMPALA-10102 Fix Impalad crashses when writting a parquet file with large rows. The crash happens due to accessing a null pointer. The reason for the accessing the null pointer is because there is no enough space left in the memory pool when the system is suffering memory scarcity, so the pool returns a null pointer when memory allocation failed, however the code doesn't verify it and then lead to a crash. The change is to use TryAllocate instead of Allocate and if it returns null pointer then returns an error status. The change is added to two places which are supposed to cause the crash. Note: The change fixes the crash issue, however in practice, there still be an OOM issue which leads the process being killed by the OS. The change doesn't fix the OOM issue, users need to have a proper configuration on the mem_limit(start-up option) to avoid the OOM happens. Test: Manually redo the test mentioned in the Jira, no crash happens. Change-Id: I0dee474cceb0c370278d290eb900c05769b23dec --- M be/src/exec/parquet/hdfs-parquet-table-writer.cc M be/src/runtime/mem-pool.h 2 files changed, 11 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/38/16638/1 -- To view, visit http://gerrit.cloudera.org:8080/16638 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I0dee474cceb0c370278d290eb900c05769b23dec Gerrit-Change-Number: 16638 Gerrit-PatchSet: 1 Gerrit-Owner: Yida Wu <wydbaggio...@gmail.com>
