[
https://issues.apache.org/jira/browse/HIVE-14979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15644934#comment-15644934
]
Peter Vary commented on HIVE-14979:
-----------------------------------
Thanks [~thejas] for your review!
As for the drawbacks of the current solution, some of them I did think about
and tired to highlight in the description of the new configuration value, some
of them I did not. Thanks for pointing those later ones out.
In both cases we try to provide resilience against temporary GC or network
issues, and a session loss or an improper shutdown will have different effect:
- GC or network issue will cause:
-- Service discovery - Shutting down of the HiveServer2 instance - please
correct me if I am wrong
-- Query locks - Possible data corruption
- Improper shutdown will cause:
-- Service discovery - Clients connecting to another server until the timeout
is reached - please correct me if I am wrong
-- Query locks - Locks persists until the timeout is reached
After this discussion I tend to agree with you that different situations call
for different configurations, so the best solution would to have the provide
the administrator the ability to match the specific needs.
What would be the default values of the new configurations?
- 20 mins for the Service discovery timeout
- 3 mins for the Lock timeout
If we agree on this, I would create a patch for it.
Thanks,
Peter
> Removing stale Zookeeper locks at HiveServer2 initialization
> ------------------------------------------------------------
>
> Key: HIVE-14979
> URL: https://issues.apache.org/jira/browse/HIVE-14979
> Project: Hive
> Issue Type: Improvement
> Components: Locking
> Reporter: Peter Vary
> Assignee: Peter Vary
> Attachments: HIVE-14979.3.patch, HIVE-14979.4.patch,
> HIVE-14979.5.patch, HIVE-14979.patch
>
>
> HiveServer2 could use Zookeeper to store token that indicate that particular
> tables are locked with the creation of persistent Zookeeper objects.
> A problem can occur when a HiveServer2 instance creates a lock on a table and
> the HiveServer2 instances crashes ("Out of Memory" for example) and the locks
> are not released in Zookeeper. This lock will then remain until it is
> manually cleared by an admin.
> There should be a way to remove stale locks at HiveServer2 initialization,
> helping the admins life.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
