[
https://issues.apache.org/jira/browse/HIVE-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162268#comment-16162268
]
Sergey Shelukhin edited comment on HIVE-17495 at 9/12/17 12:15 AM:
-------------------------------------------------------------------
Some minor comments; my main concern - I don't see prewarm improvements and it
looks like this will make prewarm even slower.
There's an issue that prewarm triggers on any access to metastore (e.g. LLAP
setup getting UDF lists, or MetaTool) and it can take forever on large
metastores. So at the very least it should be off by default and explicitly
triggered in HS2 only.
Also even for regular prewarm for slow metastore, given how it runs, either
startup, or many threads in HS2 may be blocked doing it as far as I see
(firstTime is only set after the call and there's no synchronization). HS2
should not be unusable because of cache prewarm; it should be done on a
background thread. Otherwise HS2 restarts become slow from users perspective.
Please do add "prewarm improvements" ;)
I'm strongly -0 (-0.1 :)) on piling new things onto this path before these
improvements.
was (Author: sershe):
Some minor comments; my main concern - I don't see prewarm improvements and it
looks like this will make prewarm even slower.
There's an issue that prewarm triggers on any access to metastore (e.g. LLAP
setup getting UDF lists, or MetaTool) and it can take forever on large
metastores. So at the very least it should be off by default and explicitly
triggered in HS2 only.
Also even for regular prewarm for slow metastore, given how it runs, either
startup, or many threads in HS2 may be blocked doing it as far as I see
(firstTime is only set after the call and there's no synchronization). HS2
should not be unusable because of cache prewarm; it should be done on a
background thread.
> CachedStore: prewarm improvements, refactoring and caching some aggregate
> stats
> -------------------------------------------------------------------------------
>
> Key: HIVE-17495
> URL: https://issues.apache.org/jira/browse/HIVE-17495
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Vaibhav Gumashta
> Assignee: Vaibhav Gumashta
> Attachments: HIVE-17495.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
