Joe McDonnell created IMPALA-8063:
-------------------------------------
Summary: Excessive logging from BeeswaxConnection::get_state()
bloats JUnitXML output
Key: IMPALA-8063
URL: https://issues.apache.org/jira/browse/IMPALA-8063
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.2.0
Reporter: Joe McDonnell
BeeswaxConnection has logging for each call of get_state:
{code:java}
def get_state(self, operation_handle):
LOG.info("-- getting state for operation: %s" % operation_handle)
return self.__beeswax_client.get_state(operation_handle.get_handle())
{code}
With IMPALA-7625, ImpalaTestSuite::wait_for_state() calls this more frequently:
{code:java}
def wait_for_state(self, handle, expected_state, timeout):
"""Waits for the given 'query_handle' to reach the 'expected_state'. If it
does not
reach the given state within 'timeout' seconds, the method throws an
AssertionError.
"""
start_time = time.time()
actual_state = self.client.get_state(handle)
while actual_state != expected_state and time.time() - start_time < timeout:
actual_state = self.client.get_state(handle)
if actual_state != expected_state:
raise Timeout("query '%s' did not reach expected state '%s', last known
state '%s'"
% (handle.get_handle().id, expected_state, actual_state))
{code}
When running our tests in exhaustive mode, that increases the size of the
logging significantly. For example:
{noformat}
Before this change:
$ ls -l TEST-impala-parallel.xml
-rw-rw-r-- 1 joe joe 34254745 Jan 6 23:23 TEST-impala-parallel.xml
$ grep "getting state for operation" TEST-impala-parallel.xml | wc -
l
1044
After this change:
$ ls -l TEST-impala-parallel.xml
-rw-rw-r-- 1 joe joe 159187682 Jan 9 14:51 TEST-impala-parallel.xml
$ grep "getting state for operation" TEST-impala-parallel.xml | wc -
l
1167084
{noformat}
We should reduce this logging. Bloated JUnitXML files add burden to developer
workstations and any Jenkins infrastructure parsing them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
