[
https://issues.apache.org/jira/browse/HIVE-28638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28638:
--------------------------------
Description:
File system statistics in LLAP is handled like:
1. get all statistics before task callable (contains thread local), using
FileSystem.getAllStatistics()
2. if more than one entry belongs to the same scheme, it's merged
3. run task
4. get all statistics again, subtract the pre-state, considering the
possibility of multiple Statistics object for the same scheme
this logic is currently tightly coupled with the StatsRecordingThreadPool,
which calls to LlapUtil, so it's not testable at all, and pollutes both
StatsRecordingThreadPool and LlapUtil
the whole before/after logic can be moved to a separate class, which can hold
the statistics as its state, making StatsRecordingThreadPool much cleaner
was:
File system statistics in LLAP is handled like:
1. get all statistics before task callable (contains thread local)
2. run task
3. get all statistics again, subtract the pre-state, and create counters
this logic is currently tightly coupled with the StatsRecordingThreadPool,
which calls to LlapUtil, so it's not really testable
the whole before/after logic can be moved to a separate class, making
StatsRecordingThreadPool much cleaner
> Refactor stats handling in StatsRecordingThreadPool
> ---------------------------------------------------
>
> Key: HIVE-28638
> URL: https://issues.apache.org/jira/browse/HIVE-28638
> Project: Hive
> Issue Type: Improvement
> Security Level: Public(Viewable by anyone)
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
>
> File system statistics in LLAP is handled like:
> 1. get all statistics before task callable (contains thread local), using
> FileSystem.getAllStatistics()
> 2. if more than one entry belongs to the same scheme, it's merged
> 3. run task
> 4. get all statistics again, subtract the pre-state, considering the
> possibility of multiple Statistics object for the same scheme
> this logic is currently tightly coupled with the StatsRecordingThreadPool,
> which calls to LlapUtil, so it's not testable at all, and pollutes both
> StatsRecordingThreadPool and LlapUtil
> the whole before/after logic can be moved to a separate class, which can hold
> the statistics as its state, making StatsRecordingThreadPool much cleaner
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
