[
https://issues.apache.org/jira/browse/HIVE-16024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888395#comment-15888395
]
Sergio Peña commented on HIVE-16024:
------------------------------------
Wouldn't there OOM errors if MSCK fetches a large number of partitions? This is
a typical problem in Hive when attempting to read all partitions from the
Metastore. The strict mode was added for that. Should the MSCK have a
restriction on that as well?
Should we use the {{PartitionIterable}} class to get all partitions in batches
to avoid OOM instead?
> MSCK Repair Requires nonstrict hive.mapred.mode
> -----------------------------------------------
>
> Key: HIVE-16024
> URL: https://issues.apache.org/jira/browse/HIVE-16024
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 2.2.0
> Reporter: Barna Zsombor Klara
> Assignee: Barna Zsombor Klara
> Attachments: HIVE-16024.01.patch, HIVE-16024.02.patch
>
>
> MSCK repair fails when hive.mapred.mode is set to strict
> HIVE-13788 modified the way we read up partitions for a table to improve
> performance. Unfortunately it is using PartitionPruner to load the partitions
> which in turn is checking hive.mapred.mode.
> The previous code did not check hive.mapred.mode.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
