[
https://issues.apache.org/jira/browse/HIVE-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-16085:
------------------------------------
Description:
Sometimes, we don't get correct alias/part desc/etc. from a split path because
the path has subdirectories - e.g. in HIVE-15870 (for MM tables).
We have utility methods to work around that by getting the partdesc recursively
from the parents of a path, but they are not applied consistently; I suspect
people add calls to these methods after they discover some particular path is
broken. Perhaps we should find such places proactively for path-to-part,
path-to-work, and path-to-alias maps.
This can happen much more with MM tables, but can also happen for any
recursive-path cases (incl. just a regular recursive path read enabled via an
MR config).
was:
Sometimes, we don't get correct alias/part desc/etc. from paths - e.g. in
HIVE-15870 (for MM tables).
We have methods in utils for partdescs to work around that, but they are not
applied consistently. I suspect people add calls to this method after they
discover some particular path is broken; perhaps we should find such places
proactively for path-to-part, path-to-work, and path-to-alias maps.
This can happen much more with MM tables, but can also happen for any
recursive-path cases (incl. just a regular recursive path read enabled via an
MR config).
> get path-to-alias, path-to-work etc recursively in Hive
> -------------------------------------------------------
>
> Key: HIVE-16085
> URL: https://issues.apache.org/jira/browse/HIVE-16085
> Project: Hive
> Issue Type: Task
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Attachments: HIVE-16085.WIP.patch
>
>
> Sometimes, we don't get correct alias/part desc/etc. from a split path
> because the path has subdirectories - e.g. in HIVE-15870 (for MM tables).
> We have utility methods to work around that by getting the partdesc
> recursively from the parents of a path, but they are not applied
> consistently; I suspect people add calls to these methods after they discover
> some particular path is broken. Perhaps we should find such places
> proactively for path-to-part, path-to-work, and path-to-alias maps.
> This can happen much more with MM tables, but can also happen for any
> recursive-path cases (incl. just a regular recursive path read enabled via an
> MR config).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
