[
https://issues.apache.org/jira/browse/IMPALA-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16752396#comment-16752396
]
ASF subversion and git services commented on IMPALA-7731:
---------------------------------------------------------
Commit 8c673ed6a00d1ed5c542a2311608993651bae850 in impala's branch
refs/heads/master from Lars Volker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=8c673ed ]
IMPALA-7731: Add Read/Exchange counters to profile
Selective scans (and by extension selective fragment instances)
take higher performance hits when reading data remotely. They can
be identified by a low ratio between data being transmitted vs data
being read from HDFS.
This change adds several counters to the profile to make it easier to
identify queries based on their scan instance selectivity.
* TotalBytesSent - The total number of bytes sent by a query in
exchange nodes. Does not include remote reads, data written to disk,
or data sent to the client.
* TotalScanBytesSent - The total number of bytes sent by fragment
instances that had a scan node in their plan.
* TotalInnerBytesSent - The total number of bytes sent by fragment
instances that did not have a scan node in their plan, i.e. that
received their input data from other instances through exchange node.
* ExchangeScanRatio - The ratio between TotalScanBytesSent and
TotalBytesRead, i.e. the selectivity over all fragment instances that
had a scan node in their plan. This counter is also added to each
fragment instance.
* InnerNodeSelectivityRatio - The ratio between bytes sent by instances
with a scan node in their plan and instances without a scan node in
their plan. This indicates how well the inner nodes of the execution
plan reduced the data volume.
Change-Id: Ife7ec78fe42558429c1cbe6e5eba79842bffd648
Reviewed-on: http://gerrit.cloudera.org:8080/12229
Reviewed-by: Lars Volker <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add ratio between scanned and transmitted bytes to fragment instances
> ---------------------------------------------------------------------
>
> Key: IMPALA-7731
> URL: https://issues.apache.org/jira/browse/IMPALA-7731
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Lars Volker
> Assignee: Lars Volker
> Priority: Major
> Labels: observability, supportability
> Attachments: Selective Scan Slowdowns.png
>
>
> Selective scans (and by extension selective fragment instances) take higher
> performance hits when reading data remotely. They can be identified by a low
> ratio between data being transmitted vs data being read from HDFS. To make it
> easier to spot those instances we should add this ratio to each instance and
> to the root of the execution profile.
> !Selective Scan Slowdowns.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
