[
https://issues.apache.org/jira/browse/HIVE-24337?focusedWorklogId=523962&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-523962
]
ASF GitHub Bot logged work on HIVE-24337:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Dec/20 16:04
Start Date: 14/Dec/20 16:04
Worklog Time Spent: 10m
Work Description: szlta opened a new pull request #1776:
URL: https://github.com/apache/hive/pull/1776
Changes:
- Added new method in LLAP IO API so that a cached LLAP record reader can be
created for arbitrary files
- Removed some unnecessary / inefficient code parts in LlapInputFormat and
AcidInputFormat
Functional testing: existing testing already covers this change:
- TestVectorizedOrcAcidRowBatchReader unit test covers the code paths of
this change when the feature is off (no LLAP present)
- TestMiniLlapCliDriver/acid_direct_update_delete.q qtest covers the code
paths where LLAP is present and ACID delete deltas are cached into LLAP IO
(since the feature flag is by default turned on)
Performance testing: used S3 to see performance gains on a small (6 rows)
table read that had ~20 delete events.
- query duration decreased from ~17s to ~7s
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 523962)
Time Spent: 4h 10m (was: 4h)
> Cache delete delta files in LLAP cache
> --------------------------------------
>
> Key: HIVE-24337
> URL: https://issues.apache.org/jira/browse/HIVE-24337
> Project: Hive
> Issue Type: New Feature
> Reporter: Ádám Szita
> Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> HIVE-23824 added the functionality of caching metadata part of orc files in
> LLAP cache, so that ACID reads can be faster. However the content itself
> still needs to be read in every single time. If this could be cached too,
> additional time could be saved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
