[
https://issues.apache.org/jira/browse/HIVE-14680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504324#comment-15504324
]
Sergey Shelukhin commented on HIVE-14680:
-----------------------------------------
[~sseth] I was assuming the normal block/split boundaries were k*large-ish
power of two, so this would suffice.
Apparently there's no such restriction. +-3 can affect another bit, however if
we make no assumptions about split boundaries, we cannot tell which way the 3
goes (e.g. for 31323, we don't know if it has to be consistent with 31320 or
31326). I guess we can just remove an extra bit.
> retain consistent splits /during/ (as opposed to across) LLAP failures on top
> of HIVE-14589
> -------------------------------------------------------------------------------------------
>
> Key: HIVE-14680
> URL: https://issues.apache.org/jira/browse/HIVE-14680
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-14680.01.patch, HIVE-14680.02.patch,
> HIVE-14680.patch
>
>
> see HIVE-14589.
> Basic idea (spent about 7 minutes thinking about this based on RB comment ;))
> is to return locations for all slots to HostAffinitySplitLocationProvider,
> the missing slots being inactive locations (based solely on the last slot
> actually present). For the splits mapped to these locations, fall back via
> different hash functions, or some sort of probing.
> This still doesn't handle all the cases, namely when the last slots are gone
> (consistent hashing is supposed to be good for this?); however for that we'd
> need more involved coordination between nodes or a central updater to
> indicate the number of nodes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
