[
https://issues.apache.org/jira/browse/IMPALA-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739789#comment-16739789
]
ASF subversion and git services commented on IMPALA-8063:
---------------------------------------------------------
Commit 51f30a6b9088cf3a0217a49a5792f1b41a8ad51b in impala's branch
refs/heads/master from Joe McDonnell
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=51f30a6 ]
IMPALA-8063: Add sleep to ImpalaTestSuite::wait_for_state() loop
ImpalaTestSuite::wait_for_state() loops waiting for the state
on the connection handle to reach the expected state. This loop
does not sleep, and the get_state() does some logging, so this
loop is generating a large amount of logging.
This adds a 0.5 second sleep to the loop. Running test_web_pages.py
shows a large reduction in logging.
webserver/test_web_page.py contains several tests that xfail but
still have run=True. Those tests are broken and use wait_for_state()
heavily, so disable those while IMPALA-8059 is being fixed.
Change-Id: Iaed0a9b292d431a64e22b460c92f6ef57b6abd1e
Reviewed-on: http://gerrit.cloudera.org:8080/12207
Reviewed-by: Sahil Takiar <[email protected]>
Reviewed-by: Pooja Nilangekar <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Joe McDonnell <[email protected]>
> Excessive logging from BeeswaxConnection::get_state() bloats JUnitXML output
> ----------------------------------------------------------------------------
>
> Key: IMPALA-8063
> URL: https://issues.apache.org/jira/browse/IMPALA-8063
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.2.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: broken-build
>
> BeeswaxConnection has logging for each call of get_state:
>
> {code:java}
> def get_state(self, operation_handle):
> LOG.info("-- getting state for operation: %s" % operation_handle)
> return self.__beeswax_client.get_state(operation_handle.get_handle())
> {code}
> With IMPALA-7625, ImpalaTestSuite::wait_for_state() calls this more
> frequently:
>
>
> {code:java}
> def wait_for_state(self, handle, expected_state, timeout):
> """Waits for the given 'query_handle' to reach the 'expected_state'. If it
> does not
> reach the given state within 'timeout' seconds, the method throws an
> AssertionError.
> """
> start_time = time.time()
> actual_state = self.client.get_state(handle)
> while actual_state != expected_state and time.time() - start_time < timeout:
> actual_state = self.client.get_state(handle)
> if actual_state != expected_state:
> raise Timeout("query '%s' did not reach expected state '%s', last known
> state '%s'"
> % (handle.get_handle().id, expected_state, actual_state))
> {code}
> When running our tests in exhaustive mode, that increases the size of the
> logging significantly. For example:
>
> {noformat}
> Before this change:
> $ ls -l TEST-impala-parallel.xml
> -rw-rw-r-- 1 joe joe 34254745 Jan 6 23:23 TEST-impala-parallel.xml
> $ grep "getting state for operation" TEST-impala-parallel.xml | wc -
> l
> 1044
> After this change:
> $ ls -l TEST-impala-parallel.xml
> -rw-rw-r-- 1 joe joe 159187682 Jan 9 14:51 TEST-impala-parallel.xml
> $ grep "getting state for operation" TEST-impala-parallel.xml | wc -
> l
> 1167084
> {noformat}
>
>
> We should reduce this logging. Bloated JUnitXML files add burden to developer
> workstations and any Jenkins infrastructure parsing them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
