[
https://issues.apache.org/jira/browse/IMPALA-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy updated IMPALA-7556:
--------------------------------------
Description:
For IMPALA-7543 I want to add some additional functionality to scan ranges.
However, the code of the ScanRange class is already quite messy. It handles
different types of files, does some buffer management, updates all kinds of
counters.
So, instead of complicating the code further, let's refactor the ScanRange
class a bit.
* Do the file operations in separate classes
** A new, abstract class could be invented to provide an API for file
operations, i.e. Open, ReadFromPos, Close, etc.
** Operations for local files and HDFS files could be implemented in child
classes
**
* Buffer management
** A new BufferStore class could be created
** This new class would be responsible for buffer management
*** it'd store the unused buffers
*** if possible, it would also handle the client and cached buffers as well
* Counters and metrics would be updated in the new classes
** E.g. ImpaladMetrics::IO_MGR_NUM_OPEN_FILES would be updated by the file
handling classes
was:
For IMPALA-7543 I want to add some additional functionality to scan ranges.
However, the code of the ScanRange class is already quite messy. It handles
different types of files, does some buffer management, updates all kinds of
counters.
So, instead of complicating the code further, let's refactor the ScanRange
class a bit.
> Clean up ScanRange
> ------------------
>
> Key: IMPALA-7556
> URL: https://issues.apache.org/jira/browse/IMPALA-7556
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
>
> For IMPALA-7543 I want to add some additional functionality to scan ranges.
> However, the code of the ScanRange class is already quite messy. It handles
> different types of files, does some buffer management, updates all kinds of
> counters.
> So, instead of complicating the code further, let's refactor the ScanRange
> class a bit.
> * Do the file operations in separate classes
> ** A new, abstract class could be invented to provide an API for file
> operations, i.e. Open, ReadFromPos, Close, etc.
> ** Operations for local files and HDFS files could be implemented in child
> classes
> **
> * Buffer management
> ** A new BufferStore class could be created
> ** This new class would be responsible for buffer management
> *** it'd store the unused buffers
> *** if possible, it would also handle the client and cached buffers as well
> * Counters and metrics would be updated in the new classes
> ** E.g. ImpaladMetrics::IO_MGR_NUM_OPEN_FILES would be updated by the file
> handling classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
