[
https://issues.apache.org/jira/browse/IMPALA-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18080205#comment-18080205
]
ASF subversion and git services commented on IMPALA-14847:
----------------------------------------------------------
Commit 2618531e68b7b916009e4e8ebaeb04991f52c1da in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2618531e6 ]
IMPALA-14847 (addendum): Add Iceberg's FilepathToHostsMapPB to large RPC logging
Iceberg DIRECTED exchanges use the by_node_filepath_to_hosts field
on ExecQueryFInstances. This could be an interesting component
of the size of ExecQueryFInstances RPCs for Iceberg queries.
This includes that size in the logging for large ExecQueryFInstances
RPCs when it is non-zero. This is a separate commit to make it easier
to backport the original commit.
Example output:
ExecQueryFInstances RPCs for 3 out of 3 exceed the warning threshold
of 512.00 KB. Average RPC size: 811.66 KB Shared TQueryCtx: 247.45 KB
(246.26 KB from descriptor table)
Additional information about the 3 largest RPCs:
Destination: joemcdonnell-22743:27001 Total size: 811.07 KB
Backend-specific size: 563.62 KB TExecPlanFragmentInfo: 2.98 KB
ExecQueryFInstancesRequestPB: 560.64 KB num fragments: 1
num fragment instances: 1 num scan ranges: 608
filepath to host map: 237.48 KB
Testing:
- Ran test_iceberg.py with a 512KB warning threshold and checked
the output
Change-Id: I1411aeea9351a55ed618d205734c9cad519daf8c
Reviewed-on: http://gerrit.cloudera.org:8080/24276
Reviewed-by: Joe McDonnell <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add warning logs for slow or large ExecQueryFInstances RPCs
> -----------------------------------------------------------
>
> Key: IMPALA-14847
> URL: https://issues.apache.org/jira/browse/IMPALA-14847
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Joe McDonnell
> Priority: Critical
>
> Similar to IMPALA-10702, it'd be helpful to log warnings for slow or large
> ExecQueryFInstances RPCs from coordinators. The request could be large since
> it contains the file paths and partition paths. Logging the warnings help to
> find the problematic queries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
