[
https://issues.apache.org/jira/browse/HIVE-16904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16904:
-------------------------------
Fix Version/s: (was: 3.0.0)
3.1.0
Deferring this to 3.1.0 since the branch for 3.0.0 has been cut off. Please
update the JIRA if you would like to get your patch in 3.0.0.
> during repl load for large number of partitions the metadata file can be huge
> and can lead to out of memory
> ------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-16904
> URL: https://issues.apache.org/jira/browse/HIVE-16904
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
> Affects Versions: 3.0.0
> Reporter: anishek
> Assignee: anishek
> Priority: Major
> Fix For: 3.1.0
>
>
> the metadata pertaining to a table + its partitions is stored in a single
> file, During repl load all the data is loaded in memory in one shot and then
> individual partitions processed. This can lead to huge memory overhead as the
> entire file is read in memory. try to deserialize the partition objects with
> some sort of streaming json deserializer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
