[
https://issues.apache.org/jira/browse/IMPALA-6640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-6640.
-----------------------------------
Resolution: Duplicate
> profile collection via standard Impyla usage won't have ExecSummary
> -------------------------------------------------------------------
>
> Key: IMPALA-6640
> URL: https://issues.apache.org/jira/browse/IMPALA-6640
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.11.0
> Reporter: Michael Brown
> Priority: Major
> Labels: profile
>
> It seems the Impyla (HS2 client for Python stress test) [DB
> API|https://www.python.org/dev/peps/pep-0249/] paradigm doesn't lend itself
> to the ideal collection of query profiles.
> # It seems a profile will not have an ExecSummary, End Time, or Errors
> unless the profile is requested after a CloseOperation RPC. This makes sense
> to me.
> # GetRuntimeProfile RPC must be called before the CloseSession RPC. This
> makes sense to me.
> # One can't really use standard, public methods in Impyla to do this:
> impala.hiveserver2.HiveServer2Cursor().close() handles both the
> CloseOperation and CloseSession RPCs in a single call.
> # The same is true of the cursor in a context manager __exit__ setting.
> __exit__ just calls close()
> This means we have to tap into the more private bits of Impyla to do this.
> {noformat}
> #!/usr/bin/env impala-python
> import impala.dbapi
> from impala._thrift_api import TGetRuntimeProfileReq
> with impala.dbapi.connect(host='localhost', port=21050) as conn:
> with conn.cursor() as cursor:
> cursor.execute('SELECT COUNT(2) FROM functional.alltypes')
> cursor.fetchall()
> with open('profile-before-close-operation.txt', 'w') as fh:
> fh.write(cursor.get_profile())
> # Normal cursor context would end here, but....
> # Issue CloseOperation RPC without CloseSession RPC.
> # This is impala.hiveserver2.Operation().close()
> resp = cursor._last_operation.close()
> req = TGetRuntimeProfileReq(operationHandle=cursor._last_operation.handle,
>
> sessionHandle=cursor._last_operation.session.handle)
> resp = cursor._last_operation._rpc('GetRuntimeProfile', req)
> # impala.hiveserver2.HiveServer2Cursor()._reset_state() assist
> cursor._last_operation_active = False
> with open('profile-after-close-operation.txt', 'w') as fh:
> fh.write(resp.profile)
> {noformat}
> We should think about pythonic ways to make this better for Impyla, or at
> least alter the stress test to get the profiles of the queries after
> CloseOperation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
