[
https://issues.apache.org/jira/browse/HIVE-16220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094156#comment-17094156
]
james601232 edited comment on HIVE-16220 at 4/28/20, 4:59 AM:
--------------------------------------------------------------
encounter same error
how to solved or avoid this error
51,403 instances of "org.apache.hadoop.conf.Configuration", loaded by
"sun.misc.Launcher$AppClassLoader @ 0x54f025510" occupy 5,938,537,888 (77.98%)
bytes. These instances are referenced from one instance of
"java.util.HashMap$Node[]", loaded by "<system class loader>"
Keywords
org.apache.hadoop.conf.Configuration
sun.misc.Launcher$AppClassLoader @ 0x54f025510
java.util.HashMap$Node[]
was (Author: james601232):
encounter same error
how to solved or avoid this error
!image-2020-04-28-12-58-11-734.png!
> Memory leak when creating a table using location and NameNode in HA
> -------------------------------------------------------------------
>
> Key: HIVE-16220
> URL: https://issues.apache.org/jira/browse/HIVE-16220
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 1.2.1, 3.0.0
> Environment: HDP-2.4.0.0
> HDP-3.1.0.0
> Reporter: Angel Alvarez Pascua
> Priority: Major
>
> The following simple DDL
> CREATE TABLE `test`(`field` varchar(1)) LOCATION
> 'hdfs://benderHA/apps/hive/warehouse/test'
> ends up generating a huge memory leak in the HiveServer2 service.
> After two weeks without a restart, the service stops suddenly because of
> OutOfMemory errors.
> This only happens when we're in an environment in which the NameNode is in
> HA, otherwise, nothing (so weird) happens. If the location clause is not
> present, everything is also fine.
> It seems, multiples instances of Hadoop configuration are created when we're
> in an HA environment:
> <AFTER ONE EXECUTIONS OF CREATE TABLE WITH LOCATION>
> 2.618 instances of "org.apache.hadoop.conf.Configuration", loaded by
> "sun.misc.Launcher$AppClassLoader @ 0x4d260de88"
> occupy 350.263.816 (81,66%) bytes. These instances are referenced from one
> instance of "java.util.HashMap$Node[]",
> loaded by "<system class loader>"
> <AFTER TWO EXECUTIONS OF CREATE TABLE WITH LOCATION>
> 5.216 instances of "org.apache.hadoop.conf.Configuration", loaded by
> "sun.misc.Launcher$AppClassLoader @ 0x4d260de88"
> occupy 699.901.416 (87,32%) bytes. These instances are referenced from one
> instance of "java.util.HashMap$Node[]",
> loaded by "<system class loader>"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
