[
https://issues.apache.org/jira/browse/IMPALA-14796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18089181#comment-18089181
]
ASF subversion and git services commented on IMPALA-14796:
----------------------------------------------------------
Commit effcbae94bb84abd20da7ec5742a5811b24f170c in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=effcbae94 ]
IMPALA-15095: Fix FilterTable in plan-graph.py missing a new column
IMPALA-14796 adds a new column, "Eff. Tgt. Node(s)", in the "Final
filter table" of the query profile. plan-graph.py should also be updated
to handle the new column.
This fixes the issue and adds a test to avoid future profile changes
break plan-graph.py.
Assisted-by: Composer 2.5
Change-Id: Ibe187bc209d2d4c64d2eb09c8aaa45d9f0370e45
Reviewed-on: http://gerrit.cloudera.org:8080/24450
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add "effective" column in "Final filter table" in query profile
> ---------------------------------------------------------------
>
> Key: IMPALA-14796
> URL: https://issues.apache.org/jira/browse/IMPALA-14796
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Labels: ramp-up
> Fix For: Impala 5.0.0
>
>
> In the query profile, there is a section about runtime filters, e.g., for
> TPCH-Q5:
> {noformat}
> Final filter table:
> ID Src. Node Tgt. Node(s) Target type Partition filter Pending
> (Expected) First arrived Completed Enabled Bloom Size Est fpp Min
> value Max value In-list size
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> 10 6 2 LOCAL false
> 0 (3) N/A N/A true 1.00 MB 2.06e-06
>
> 8 7 1 REMOTE false
> 0 (3) 392.242ms 393.338ms true 1.00 MB 2.06e-06
>
> 5 8 2 LOCAL false
> 0 (3) N/A N/A true 1.00 MB 3.54e-11
>
> 4 8 0 REMOTE false
> 0 (3) 351.647ms 351.978ms true 1.00 MB 7.53e-16
>
> 2 9 0, 3 REMOTE, REMOTE false, false
> 0 (3) 347.219ms 347.494ms true 1.00 MB 7.53e-16
>
> 0 10 4 REMOTE false
> 0 (3) 342.907ms 343.293ms true 1.00 MB 2.79e-17{noformat}
> It'd be helpful to add a boolean column "effective" to show whether the
> filter actually rejects any data (rows/RowGroups/splits/files).
> Currently, we have to check the "rejected" counters of the ScanNodes, e.g.,
> {noformat}
> Filter 2 (1.00 MB):
> - Files processed: 0 (0)
> - Files rejected: 0 (0)
> - Files total: 0 (0)
> - RowGroups processed: 1 (1)
> - RowGroups rejected: 0 (0)
> - RowGroups total: 1 (1)
> - Rows processed: 150.00K (150000)
> - Rows rejected: 119.82K (119817)
> - Rows total: 150.00K (150000)
> - Splits processed: 0 (0)
> - Splits rejected: 0 (0)
> - Splits total: 0 (0)
> Filter 4 (1.00 MB):
> - Files processed: 0 (0)
> - Files rejected: 0 (0)
> - Files total: 0 (0)
> - RowGroups processed: 1 (1)
> - RowGroups rejected: 0 (0)
> - RowGroups total: 1 (1)
> - Rows processed: 16.38K (16384)
> - Rows rejected: 0 (0)
> - Rows total: 30.18K (30183)
> - Splits processed: 0 (0)
> - Splits rejected: 0 (0)
> - Splits total: 0 (0){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
