[
https://issues.apache.org/jira/browse/IMPALA-8114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Volker reassigned IMPALA-8114:
-----------------------------------
Assignee: Lars Volker (was: Tim Armstrong)
> Build test failure in test_breakpad.py
> --------------------------------------
>
> Key: IMPALA-8114
> URL: https://issues.apache.org/jira/browse/IMPALA-8114
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.1.0
> Reporter: Paul Rogers
> Assignee: Lars Volker
> Priority: Blocker
>
> Recent builds have failed due to a failure in {{test_breakpad.py}}. Assigning
> to Tim as the person who most recently touched this file.
> Test output:
> {noformat}
> 09:04:35 ==================================== ERRORS
> ====================================
> 09:04:35 ___ ERROR at teardown of
> TestBreakpadExhaustive.test_minidump_cleanup_thread ___
> 09:04:35 custom_cluster/test_breakpad.py:49: in teardown_method
> 09:04:35 self.kill_cluster(SIGKILL)
> 09:04:35 custom_cluster/test_breakpad.py:80: in kill_cluster
> 09:04:35 self.kill_processes(processes, signal)
> 09:04:35 custom_cluster/test_breakpad.py:85: in kill_processes
> 09:04:35 process.kill(signal)
> 09:04:35 common/impala_cluster.py:330: in kill
> 09:04:35 assert 0, "No processes %s found" % self.cmd
> 09:04:35 E AssertionError: No processes
> ['/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/be/build/latest/service/impalad',
> '-kudu_client_rpc_timeout_ms', '0', '-kudu_master_hosts', 'localhost',
> '--mem_limit=12884901888', '-logbufsecs=5', '-v=1', '-max_log_files=0',
> '-log_filename=impalad',
> '-log_dir=/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/logs/custom_cluster_tests',
> '-beeswax_port=21000', '-hs2_port=21050', '-be_port=22000',
> '-krpc_port=27000', '-state_store_subscriber_port=23000',
> '-webserver_port=25000', '-max_minidumps=2', '-logbufsecs=1',
> '-minidump_path=/tmp/tmpKaSw_w', '--default_query_options='] found
> {noformat}
> Distilled {{TEST-impala-custom-cluster.xml}} output:
> {noformat}
> -- 2019-01-23 08:00:43,585 INFO MainThread: Found 3 impalad/1
> statestored/1 catalogd process(es)
> …
> -- 2019-01-23 08:00:43,667 INFO MainThread: Killing:
> /data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/be/build/latest/service/statestored
> -logbufsecs=5 -v=1 -max_log_files=0 -log_filename=statestored
> -log_dir=/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/logs/custom_cluster_tests
> -max_minidumps=2 -logbufsecs=1 -minidump_path=/tmp/tmpKaSw_w (PID: 16809)
> with signal 10
> -- 2019-01-23 08:00:43,692 INFO MainThread: Found 6 impalad/1
> statestored/1 catalogd process(es)
> ...
> E AssertionError: No processes
> ['/data/jenkins/workspace/impala-cdh6.x-exhaustive-release/repos/Impala/be/build/latest/service/impalad
> {noformat}
> Notice that the main thread appaars to be killing statestore, but fails to
> kill impalad. Notice that a message appears that says that all impalads are
> running in the midst of the code that tries to shut down the cluster. Is this
> test multi-threaded? Is there more than one “main thread” Are these main
> threads working at cross purposes? What recent change may have caused this?
> Also, looks like the script is sending signal 10 (SIGUSR1) while the
> statestore (in its log) says it got a SIGTERM (15):
> {noformat}
> I0123 08:00:44.086009 16868 thrift-client.cc:78] Couldn't open transport for
> impala-ec2-centoCaught signal: SIGTERM. Daemon will exit.
> {noformat}
> Not terribly familiar with this area of the product, so bumping it over to
> the BE team.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
