[
https://issues.apache.org/jira/browse/HIVE-14680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-14680:
------------------------------------
Description:
see HIVE-14589.
Basic idea (spent about 7 minutes thinking about this based on RB comment ;))
is to return locations for all slots to HostAffinitySplitLocationProvider, the
missing slots being inactive locations (based solely on the last slot actually
present). For the splits mapped to these locations, fall back via different
hash functions, or some sort of probing.
This still doesn't handle all the cases, namely when the last slots are gone
(consistent hashing is supposed to be good for this?); however for that we'd
need more involved coordination between nodes or a central updater to indicate
the number of nodes
was:
see HIVE-14589.
Basic idea (spent about 7 minutes thinking about this based on RB comment ;))
is to return locations for all slots to HostAffinitySplitLocationProvider, the
missing slots being inactive locations (based solely on the last slot actually
present). For the splits mapped to these locations, fall back via different
hash functions, or some sort of probing.
This still doesn't handle all the cases, namely when the last slots are gone
(consistent hashing is supposed to be good for this?); however for that we'd
need more involved coordination between nodes or a central updater to indicate
the number of slots
> retain consistent splits /during/ (as opposed to across) LLAP failures on top
> of HIVE-14589
> -------------------------------------------------------------------------------------------
>
> Key: HIVE-14680
> URL: https://issues.apache.org/jira/browse/HIVE-14680
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-14680.patch
>
>
> see HIVE-14589.
> Basic idea (spent about 7 minutes thinking about this based on RB comment ;))
> is to return locations for all slots to HostAffinitySplitLocationProvider,
> the missing slots being inactive locations (based solely on the last slot
> actually present). For the splits mapped to these locations, fall back via
> different hash functions, or some sort of probing.
> This still doesn't handle all the cases, namely when the last slots are gone
> (consistent hashing is supposed to be good for this?); however for that we'd
> need more involved coordination between nodes or a central updater to
> indicate the number of nodes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
