Hello Thomas Tauber-Marshall, Kurt Deschler, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17183
to look at the new patch set (#2).
Change subject: IMPALA-10552: Support external frontends supplying timeline for
profile
......................................................................
IMPALA-10552: Support external frontends supplying timeline for profile
- Add EXTERNAL_FRONTEND as a client session type
- Use EXTERNAL_FRONTEND session type for clients connected to
external frontend interface.
- Rename Query Timeline to Impala Backend Timeline for external
frontends
- the query timeline is no longer an end to end timeline when
executing a plan from an external frontend
- External frontends can provide timeline information through a
TExecRequest by filling in the timeline field with a valid
TEventSequence
- The frontend timeline and backend timeline are completely separate
entities, meaning there is no overall attempt to capture the timing
end to end
- This is due to the fact that the frontend and Impala may not share
the same time source (or even machine).
- It is safe to add together the backend + frontend timeline times
to get a rough idea how long the query took end to end to execute,
but keep in mind that this number does not capture the time it
took the frontend to send the plan to the backend (Impala) nor does
it capture how long it took the end user to read the results.
Example timeline with external frontend:
Frontend Timeline: 3s016ms
- Analysis finished: 1s130ms (1s130ms)
- Calcite plan generated: 2s170ms (1s040ms)
- Metadata load started: 2s245ms (74.486ms)
- Metadata load finished. loaded-tables=1: 2s654ms (409.847ms)
- Single node plan created: 2s726ms (71.659ms)
- Runtime filters computed: 2s756ms (30.000ms)
- Distributed plan created: 2s761ms (5.265ms)
- Execution request created: 2s890ms (128.387ms)
- Impala plan generated: 2s891ms (1.508ms)
- Planning finished: 2s893ms (1.894ms)
- Submitted query: 3s016ms (122.377ms)
Impala Backend Timeline: 79.998ms
- Query submitted: 0.000ns (0.000ns)
- Submit for admission: 0.000ns (0.000ns)
- Completed admission: 0.000ns (0.000ns)
- Ready to start on 1 backends: 3.999ms (3.999ms)
- All 1 execution backends (2 fragment instances) started: 7.999ms
(3.999ms)
- Rows available: 55.999ms (47.999ms)
- Execution cancelled: 79.998ms (23.999ms)
- Released admission control resources: 79.998ms (0.000ns)
- Unregister query: 79.998ms (0.000ns)
Testing done:
- Manual inspection of profiles on the Impala web UI
- test_hs2.py
- test_tpch_queries.py
- test_tpcds_queries.py::TestTpcdsDecimalV2Query
Co-authored-by: Kurt Deschler <[email protected]>
Change-Id: I2b3692b4118ea23c0f9f8ec4bcc27b0b68bb32ec
---
M be/src/service/client-request-state.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-server.cc
M common/thrift/Query.thrift
4 files changed, 30 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/17183/2
--
To view, visit http://gerrit.cloudera.org:8080/17183
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2b3692b4118ea23c0f9f8ec4bcc27b0b68bb32ec
Gerrit-Change-Number: 17183
Gerrit-PatchSet: 2
Gerrit-Owner: John Sherman <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: John Sherman <[email protected]>
Gerrit-Reviewer: Kurt Deschler <[email protected]>
Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]>
