[
https://issues.apache.org/jira/browse/HIVE-19820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534208#comment-16534208
]
Sergey Shelukhin commented on HIVE-19820:
-----------------------------------------
So after fixing that, the next issue not covered by the new scheme is actually
that analyze should actually be able to rewrite invalid stats (duh), whereas
the new scheme doesn't allow it.
At the same time if parallel inserts are present, analyze should not mark stats
as valid... it's easy to tell apart the cases where writeID is set, but it's
not so easy to tell apart the cases where someone just unsets the flag without
having a write ID. Then we come back and we don't know if the flag was unset
because stats were invalid before analyze and we should update stats, or
whether the flag was unset by some silly parallel operation (could be anything
from truncate to simply an insert with auto-update disabled by hand), so our
analyze is actually moot.
In the previous scheme having the whole BLOB in the table solved this because
the blob was validated on read side and would trivially be marked as invalid.
I think I know how to solve this (analyze would update stats in the beginning
to say they are valid, and to its own uncommitted write ID), but I'm not sure
yet it will cover all the cases and what races are possible.
> add ACID stats support to background stats updater
> --------------------------------------------------
>
> Key: HIVE-19820
> URL: https://issues.apache.org/jira/browse/HIVE-19820
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Priority: Major
> Attachments: HIVE-19820.01-master-txnstats.patch,
> HIVE-19820.02-master-txnstats.patch, HIVE-19820.03-master-txnstats.patch,
> HIVE-19820.04-master-txnstats.patch
>
>
> Follow-up from HIVE-19418.
> Right now it checks whether stats are valid in an old-fashioned way... and
> also gets ACID state, and discards it without using.
> When ACID stats are implemented, ACID state needs to be used to do
> version-aware valid stats checks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
