[
https://issues.apache.org/jira/browse/HIVE-20948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-20948:
----------------------------------
Description:
Once HIVE-20823 is committed, we should investigate if it's possible to have
compactor write directly to base_x_cZ or delta_x_y_cZ.
For query based compaction: can we control location of temp table dir? We
support external temp tables so this may work but we'd need to have non-acid
insert create files with {{bucket_xxxxx}} names.
For MR/Tez/LLAP based (should this be done at all?), need to figure out how
retries of tasks will work. Just like we currently generate an MR job to
compact, we should be able to generate a Tez job.
was:
Once HIVE-20823 is committed, we should investigate if it's possible to have
compactor write directly to base_x_cZ or delta_x_y_cZ.
For query based compaction: can we control location of temp table dir? We
support external temp tables so this may work but we'd need to have non-acid
insert create files with {{bucket_xxxxx}} names.
For MR/Tez/LLAP based (should this be done at all?), need to figure out how
retries of tasks will work.
> Eliminate file rename in compactor
> ----------------------------------
>
> Key: HIVE-20948
> URL: https://issues.apache.org/jira/browse/HIVE-20948
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 4.0.0
> Reporter: Eugene Koifman
> Priority: Major
>
> Once HIVE-20823 is committed, we should investigate if it's possible to have
> compactor write directly to base_x_cZ or delta_x_y_cZ.
> For query based compaction: can we control location of temp table dir? We
> support external temp tables so this may work but we'd need to have non-acid
> insert create files with {{bucket_xxxxx}} names.
>
> For MR/Tez/LLAP based (should this be done at all?), need to figure out how
> retries of tasks will work. Just like we currently generate an MR job to
> compact, we should be able to generate a Tez job.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
