xuchuanyin created CARBONDATA-3026:
--------------------------------------
Summary: clear expired property that may cause GC problem
Key: CARBONDATA-3026
URL: https://issues.apache.org/jira/browse/CARBONDATA-3026
Project: CarbonData
Issue Type: Bug
Components: data-load
Reporter: xuchuanyin
Assignee: xuchuanyin
During data loading, we will write some temp files (sort temp
files and temp fact data files) in some locations. In currently
implementation, we will add the locations to the CarbonProperties and
associated it with a special key that refers to the data loading.
After data loading, the temp locations are cleared, but the added
property is still remain in the CarbonProperties and never to be cleared.
This will cause the CarbonProperties object growing bigger and bigger
and lead to OOM problems if the thrift-server is a long time running
service. A local test shows that after adding different properties for
11 Billion times, the OOM happens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
