[
https://issues.apache.org/jira/browse/HIVE-19439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466029#comment-16466029
]
Vihang Karajgaonkar commented on HIVE-19439:
--------------------------------------------
copying this from HIVE-19388 for reference:
{quote}
As for your observation about spark_vectorized_dynamic_partition_pruning.q,
seems that's indeed another bug. The task fails during MapWork initialization.
When we retry the task, we retrieve the MapWork from cache. At this point, some
operator's state is State.INIT, although the previous initialization actually
failed. So initialization is skipped and the task somehow finishes
successfully. I think one way to fix it is to clear the work cache when
initialization fails.
{quote}
Hi [~lirui] Can you please point me to the code which retries the task? Thanks!
> MapWork shouldn't be reused when Spark task fails during initialization
> -----------------------------------------------------------------------
>
> Key: HIVE-19439
> URL: https://issues.apache.org/jira/browse/HIVE-19439
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Rui Li
> Priority: Major
>
> Issue identified in HIVE-19388. When a Spark task fails during initializing
> the map operator, the task is retried with the same MapWork retrieved from
> cache. This can be problematic because the MapWork may be partially
> initialized, e.g. some operators are already in INIT state.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
