[ https://issues.apache.org/jira/browse/IMPALA-6241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-6241. ----------------------------------- Resolution: Fixed Fix Version/s: Impala 2.11.0 IMPALA-6241: timeout in admission control test under ASAN The fix for IMPALA-6241 is to increase the timeout for all slow builds. While testing that fix, I discovered that the ASAN build detection logic was failing silently, resulting in it assuming that it was testing a DEBUG build. The error was: Unexpected DW_AT_name in first CU: /data/jenkins/workspace/verify-impala-toolchain-package-build/label/ec2-package-ubuntu-16-04/toolchain/source/llvm/llvm-3.9.1.src/projects/compiler-rt/lib/asan/asan_preinit.cc; choosing DEBUG The fix for that issue is to remove the build type detection heuristic and instead just write a file with the build type as part of the build process. Testing: Before this change I was able to reproduce locally every 5-10 test iterations. After this change I haven't seen it reproduce. Change-Id: Ia4ed949cac99b9925f72e19e4adaa2ead370b536 Reviewed-on: http://gerrit.cloudera.org:8080/8652 Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com> Tested-by: Impala Public Jenkins > Timeout in TestAdmissionControllerStress.test_mem_limit under ASAN > ------------------------------------------------------------------ > > Key: IMPALA-6241 > URL: https://issues.apache.org/jira/browse/IMPALA-6241 > Project: IMPALA > Issue Type: Bug > Affects Versions: Impala 2.11.0 > Reporter: Tianyi Wang > Assignee: Tim Armstrong > Priority: Blocker > Labels: broken-build > Fix For: Impala 2.11.0 > > > In an ASAN build, test_mem_limit times out in end_admitted_queries(). It's > probably related to IMPALA-6227 > {noformat} > def end_admitted_queries(self, num_queries): > """ > Requests each admitted query to end its query. > """ > assert len(self.executing_threads) >= num_queries > LOG.debug("Requesting {0} clients to end queries".format(num_queries)) > > # Request admitted clients to end their queries > current_executing_queries = [] > for i in xrange(num_queries): > # pop() is thread-safe, it's OK if another thread is appending > concurrently. > thread = self.executing_threads.pop(0) > LOG.debug("Cancelling query %s", thread.query_num) > assert thread.query_state == 'ADMITTED' > current_executing_queries.append(thread) > thread.query_state = 'REQUEST_QUERY_END' > > # Wait for the queries to end > start_time = time() > while True: > all_done = True > for thread in self.all_threads: > if thread.query_state == 'REQUEST_QUERY_END': > all_done = False > if all_done: > break > > assert (time() - start_time < STRESS_TIMEOUT),\ > "Timed out waiting %s seconds for query end" % (STRESS_TIMEOUT,) > E AssertionError: Timed out waiting 60 seconds for query end > E assert (1511608615.681211 - 1511608555.6205289) < 60 > E + where 1511608615.681211 = time() > custom_cluster/test_admission_controller.py:623: AssertionError > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)