See <https://builds.apache.org/job/Aurora/1937/display/redirect?page=changes>
Changes: [serb] Allow for injection of custom OfferSets, removed OfferOrder and ------------------------------------------ [...truncated 581.50 KB...] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_create_ioerror <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 98%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_docker_directory_sandbox_create_ioerror <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 98%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_destroy_ioerror <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 98%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_verify_network_files <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 98%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_user_name_exists <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_gid_exists <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_verify_user_match <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_filesystem_sandbox_mounts_paths_source_is_file <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_uid_exists <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_filesystem_sandbox_mounts_paths <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_verify_group_match <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_filesystem_sandbox_no_volumes <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [ 99%] src/test/python/apache/aurora/executor/common/test_sandbox.py::test_group_name_exists <- .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/common/test_sandbox.py PASSED [100%] generated xml file: <https://builds.apache.org/job/Aurora/ws/.pants.d/test/pytest/aaf4d108c31293299a0839bdc404a91802f80937/junitxml/TEST-aaf4d108c31293299a0839bdc404a91802f80937.xml> ==================== FAILURES ==================== _____ TestThermosExecutor.test_basic_as_job ______ self = <apache.aurora.executor.test_thermos_executor.TestThermosExecutor object at 0x7faa59f7d690> def test_basic_as_job(self): proxy_driver = ProxyDriver() with temporary_dir() as tempdir: te = AuroraExecutor( runner_provider=make_provider(tempdir), sandbox_provider=DefaultTestSandboxProvider(), status_providers=[HealthCheckerProvider()]) te.launchTask(proxy_driver, make_task(MESOS_JOB(task=HELLO_WORLD), instanceId=0)) te.runner_started.wait() while te._status_manager is None: time.sleep(0.1) te.terminated.wait() tm = TaskMonitor(tempdir, task_id=HELLO_WORLD_TASK_ID) runner_state = tm.get_state() assert 'hello_world_hello_world-001' in runner_state.processes, ( 'Could not find processes, got: %s' % ' '.join(runner_state.processes)) updates = proxy_driver.method_calls['sendStatusUpdate'] > assert len(updates) == 3 E assert 2 == 3 E + where 2 = len([((task_id {\n value: "hello_world-001"\n}\nstate: TASK_STARTING\n,), {}), ((task_id {\n value: "hello_world-001"\n}\nstate: TASK_FINISHED\nmessage: "Task finished."\n,), {})]) .pants.d/pyprep/sources/968fc9b01c64005fd3afb6de6a11e96de3acd375/apache/aurora/executor/test_thermos_executor.py:362: AssertionError -------------- Captured stderr call -------------- INFO] Executor [None]: TaskInfo: name: "hello_world" task_id { value: "hello_world-001" } data: "\013\000\001\000\000\000\017hello_world-001\014\000\004\014\000\031\013\000\001\000\000\000\016AuroraExecutor\013\000\002\000\000\004\314{\"priority\": 0, \"health_check_config\": {\"health_checker\": {\"http\": {\"expected_response_code\": 0, \"endpoint\": \"/health\", \"expected_response\": \"ok\"}}, \"min_consecutive_successes\": 1, \"initial_interval_secs\": 15.0, \"max_consecutive_failures\": 0, \"timeout_secs\": 1.0, \"interval_secs\": 10.0}, \"name\": \"does_not_matter\", \"service\": false, \"update_config\": {\"wait_for_batch_completion\": false, \"batch_size\": 1, \"watch_secs\": 45, \"rollback_on_failure\": true, \"max_per_shard_failures\": 0, \"max_total_failures\": 0}, \"max_task_failures\": 1, \"cron_collision_policy\": \"KILL_EXISTING\", \"enable_hooks\": false, \"instances\": 1, \"task\": {\"processes\": [{\"daemon\": false, \"name\": \"hello_world_{{thermos.task_id}}\", \"ephemeral\": false, \"max_failures\": 1, \"min_duration\": 5, \"cmdline\": \"echo hello world\", \"final\": false}], \"name\": \"hello_world\", \"finalization_wait\": 30, \"max_failures\": 1, \"max_concurrency\": 0, \"resources\": {\"gpu\": 0, \"disk\": 33554432, \"ram\": 16777216, \"cpu\": 1.0}, \"constraints\": []}, \"production\": false, \"role\": \"jenkins\", \"lifecycle\": {\"http\": {\"graceful_shutdown_endpoint\": \"/quitquitquit\", \"graceful_shutdown_wait_secs\": 5, \"port\": \"health\", \"shutdown_wait_secs\": 5, \"shutdown_endpoint\": \"/abortabortabort\"}}, \"metadata\": []}\000\014\000\034\013\000\001\000\000\000\007jenkins\013\000\002\000\000\000\003env\013\000\003\000\000\000\004name\000\014\000\035\014\000\001\000\000\000\r\000\005\013\010\000\000\000\000\010\000\006\000\000\000\000\000" INFO] Executor [None]: TaskInfo: name: "hello_world" task_id { value: "hello_world-001" } data: "\013\000\001\000\000\000\017hello_world-001\014\000\004\014\000\031\013\000\001\000\000\000\016AuroraExecutor\013\000\002\000\000\004\314{\"priority\": 0, \"health_check_config\": {\"health_checker\": {\"http\": {\"expected_response_code\": 0, \"endpoint\": \"/health\", \"expected_response\": \"ok\"}}, \"min_consecutive_successes\": 1, \"initial_interval_secs\": 15.0, \"max_consecutive_failures\": 0, \"timeout_secs\": 1.0, \"interval_secs\": 10.0}, \"name\": \"does_not_matter\", \"service\": false, \"update_config\": {\"wait_for_batch_completion\": false, \"batch_size\": 1, \"watch_secs\": 45, \"rollback_on_failure\": true, \"max_per_shard_failures\": 0, \"max_total_failures\": 0}, \"max_task_failures\": 1, \"cron_collision_policy\": \"KILL_EXISTING\", \"enable_hooks\": false, \"instances\": 1, \"task\": {\"processes\": [{\"daemon\": false, \"name\": \"hello_world_{{thermos.task_id}}\", \"ephemeral\": false, \"max_failures\": 1, \"min_duration\": 5, \"cmdline\": \"echo hello world\", \"final\": false}], \"name\": \"hello_world\", \"finalization_wait\": 30, \"max_failures\": 1, \"max_concurrency\": 0, \"resources\": {\"gpu\": 0, \"disk\": 33554432, \"ram\": 16777216, \"cpu\": 1.0}, \"constraints\": []}, \"production\": false, \"role\": \"jenkins\", \"lifecycle\": {\"http\": {\"graceful_shutdown_endpoint\": \"/quitquitquit\", \"graceful_shutdown_wait_secs\": 5, \"port\": \"health\", \"shutdown_wait_secs\": 5, \"shutdown_endpoint\": \"/abortabortabort\"}}, \"metadata\": []}\000\014\000\034\013\000\001\000\000\000\007jenkins\013\000\002\000\000\000\003env\013\000\003\000\000\000\004name\000\014\000\035\014\000\001\000\000\000\r\000\005\013\010\000\000\000\000\010\000\006\000\000\000\000\000" INFO] Executor [None]: TaskInfo: name: "hello_world" task_id { value: "hello_world-001" } data: "\013\000\001\000\000\000\017hello_world-001\014\000\004\014\000\031\013\000\001\000\000\000\016AuroraExecutor\013\000\002\000\000\004\314{\"priority\": 0, \"health_check_config\": {\"health_checker\": {\"http\": {\"expected_response_code\": 0, \"endpoint\": \"/health\", \"expected_response\": \"ok\"}}, \"min_consecutive_successes\": 1, \"initial_interval_secs\": 15.0, \"max_consecutive_failures\": 0, \"timeout_secs\": 1.0, \"interval_secs\": 10.0}, \"name\": \"does_not_matter\", \"service\": false, \"update_config\": {\"wait_for_batch_completion\": false, \"batch_size\": 1, \"watch_secs\": 45, \"rollback_on_failure\": true, \"max_per_shard_failures\": 0, \"max_total_failures\": 0}, \"max_task_failures\": 1, \"cron_collision_policy\": \"KILL_EXISTING\", \"enable_hooks\": false, \"instances\": 1, \"task\": {\"processes\": [{\"daemon\": false, \"name\": \"hello_world_{{thermos.task_id}}\", \"ephemeral\": false, \"max_failures\": 1, \"min_duration\": 5, \"cmdline\": \"echo hello world\", \"final\": false}], \"name\": \"hello_world\", \"finalization_wait\": 30, \"max_failures\": 1, \"max_concurrency\": 0, \"resources\": {\"gpu\": 0, \"disk\": 33554432, \"ram\": 16777216, \"cpu\": 1.0}, \"constraints\": []}, \"production\": false, \"role\": \"jenkins\", \"lifecycle\": {\"http\": {\"graceful_shutdown_endpoint\": \"/quitquitquit\", \"graceful_shutdown_wait_secs\": 5, \"port\": \"health\", \"shutdown_wait_secs\": 5, \"shutdown_endpoint\": \"/abortabortabort\"}}, \"metadata\": []}\000\014\000\034\013\000\001\000\000\000\007jenkins\013\000\002\000\000\000\003env\013\000\003\000\000\000\004name\000\014\000\035\014\000\001\000\000\000\r\000\005\013\010\000\000\000\000\010\000\006\000\000\000\000\000" INFO] Executor [None]: launchTask got task: hello_world:hello_world-001 INFO] Executor [None]: launchTask got task: hello_world:hello_world-001 INFO] Executor [None]: launchTask got task: hello_world:hello_world-001 INFO] Executor []: Assigned task: AssignedTask(task=TaskConfig(isService=None, contactEmail=None, taskLinks=None, container=Container(docker=None, mesos=MesosContainer(image=None, volumes=None)), mesosFetcherUris=None, partitionPolicy=None, executorConfig=ExecutorConfig(data=u'{"priority": 0, "health_check_config": {"health_checker": {"http": {"expected_response_code": 0, "endpoint": "/health", "expected_response": "ok"}}, "min_consecutive_successes": 1, "initial_interval_secs": 15.0, "max_consecutive_failures": 0, "timeout_secs": 1.0, "interval_secs": 10.0}, "name": "does_not_matter", "service": false, "update_config": {"wait_for_batch_completion": false, "batch_size": 1, "watch_secs": 45, "rollback_on_failure": true, "max_per_shard_failures": 0, "max_total_failures": 0}, "max_task_failures": 1, "cron_collision_policy": "KILL_EXISTING", "enable_hooks": false, "instances": 1, "task": {"processes": [{"daemon": false, "name": "hello_world_{{thermos.task_id}}", "ephemeral": false, "max_failures": 1, "min_duration": 5, "cmdline": "echo hello world", "final": false}], "name": "hello_world", "finalization_wait": 30, "max_failures": 1, "max_concurrency": 0, "resources": {"gpu": 0, "disk": 33554432, "ram": 16777216, "cpu": 1.0}, "constraints": []}, "production": false, "role": "jenkins", "lifecycle": {"http": {"graceful_shutdown_endpoint": "/quitquitquit", "graceful_shutdown_wait_secs": 5, "port": "health", "shutdown_wait_secs": 5, "shutdown_endpoint": "/abortabortabort"}}, "metadata": []}', name=u'AuroraExecutor'), maxTaskFailures=None, priority=None, tier=None, job=JobKey(environment=u'env', role=u'jenkins', name=u'name'), production=None, owner=None, metadata=None, resources=None, constraints=None), taskId=u'hello_world-001', instanceId=0, assignedPorts={}, slaveHost=None, slaveId=None) INFO] Executor []: Assigned task: AssignedTask(task=TaskConfig(isService=None, contactEmail=None, taskLinks=None, container=Container(docker=None, mesos=MesosContainer(image=None, volumes=None)), mesosFetcherUris=None, partitionPolicy=None, executorConfig=ExecutorConfig(data=u'{"priority": 0, "health_check_config": {"health_checker": {"http": {"expected_response_code": 0, "endpoint": "/health", "expected_response": "ok"}}, "min_consecutive_successes": 1, "initial_interval_secs": 15.0, "max_consecutive_failures": 0, "timeout_secs": 1.0, "interval_secs": 10.0}, "name": "does_not_matter", "service": false, "update_config": {"wait_for_batch_completion": false, "batch_size": 1, "watch_secs": 45, "rollback_on_failure": true, "max_per_shard_failures": 0, "max_total_failures": 0}, "max_task_failures": 1, "cron_collision_policy": "KILL_EXISTING", "enable_hooks": false, "instances": 1, "task": {"processes": [{"daemon": false, "name": "hello_world_{{thermos.task_id}}", "ephemeral": false, "max_failures": 1, "min_duration": 5, "cmdline": "echo hello world", "final": false}], "name": "hello_world", "finalization_wait": 30, "max_failures": 1, "max_concurrency": 0, "resources": {"gpu": 0, "disk": 33554432, "ram": 16777216, "cpu": 1.0}, "constraints": []}, "production": false, "role": "jenkins", "lifecycle": {"http": {"graceful_shutdown_endpoint": "/quitquitquit", "graceful_shutdown_wait_secs": 5, "port": "health", "shutdown_wait_secs": 5, "shutdown_endpoint": "/abortabortabort"}}, "metadata": []}', name=u'AuroraExecutor'), maxTaskFailures=None, priority=None, tier=None, job=JobKey(environment=u'env', role=u'jenkins', name=u'name'), production=None, owner=None, metadata=None, resources=None, constraints=None), taskId=u'hello_world-001', instanceId=0, assignedPorts={}, slaveHost=None, slaveId=None) INFO] Executor []: Assigned task: AssignedTask(task=TaskConfig(isService=None, contactEmail=None, taskLinks=None, container=Container(docker=None, mesos=MesosContainer(image=None, volumes=None)), mesosFetcherUris=None, partitionPolicy=None, executorConfig=ExecutorConfig(data=u'{"priority": 0, "health_check_config": {"health_checker": {"http": {"expected_response_code": 0, "endpoint": "/health", "expected_response": "ok"}}, "min_consecutive_successes": 1, "initial_interval_secs": 15.0, "max_consecutive_failures": 0, "timeout_secs": 1.0, "interval_secs": 10.0}, "name": "does_not_matter", "service": false, "update_config": {"wait_for_batch_completion": false, "batch_size": 1, "watch_secs": 45, "rollback_on_failure": true, "max_per_shard_failures": 0, "max_total_failures": 0}, "max_task_failures": 1, "cron_collision_policy": "KILL_EXISTING", "enable_hooks": false, "instances": 1, "task": {"processes": [{"daemon": false, "name": "hello_world_{{thermos.task_id}}", "ephemeral": false, "max_failures": 1, "min_duration": 5, "cmdline": "echo hello world", "final": false}], "name": "hello_world", "finalization_wait": 30, "max_failures": 1, "max_concurrency": 0, "resources": {"gpu": 0, "disk": 33554432, "ram": 16777216, "cpu": 1.0}, "constraints": []}, "production": false, "role": "jenkins", "lifecycle": {"http": {"graceful_shutdown_endpoint": "/quitquitquit", "graceful_shutdown_wait_secs": 5, "port": "health", "shutdown_wait_secs": 5, "shutdown_endpoint": "/abortabortabort"}}, "metadata": []}', name=u'AuroraExecutor'), maxTaskFailures=None, priority=None, tier=None, job=JobKey(environment=u'env', role=u'jenkins', name=u'name'), production=None, owner=None, metadata=None, resources=None, constraints=None), taskId=u'hello_world-001', instanceId=0, assignedPorts={}, slaveHost=None, slaveId=None) INFO] Executor []: Updating hello_world-001 => STARTING INFO] Executor []: Updating hello_world-001 => STARTING INFO] Executor []: Updating hello_world-001 => STARTING INFO] Executor []: Reason: None INFO] Executor []: Reason: None INFO] Executor []: Reason: None DEBUG] DirectorySandbox: mkdir /tmp/tmp8iJqey DEBUG] DirectorySandbox: mkdir /tmp/tmp8iJqey DEBUG] DirectorySandbox: mkdir /tmp/tmp8iJqey DEBUG] DirectorySandbox: chown jenkins:jenkins /tmp/tmp8iJqey DEBUG] DirectorySandbox: chown jenkins:jenkins /tmp/tmp8iJqey DEBUG] DirectorySandbox: chown jenkins:jenkins /tmp/tmp8iJqey DEBUG] DirectorySandbox: chmod 700 /tmp/tmp8iJqey DEBUG] DirectorySandbox: chmod 700 /tmp/tmp8iJqey DEBUG] DirectorySandbox: chmod 700 /tmp/tmp8iJqey WARN] Could not read from checkpoint /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner WARN] Could not read from checkpoint /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner WARN] Could not read from checkpoint /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner INFO] Forking off runner with cmdline: /usr/bin/python2.7 /tmp/tmp0fSLLb/thermos_runner.pex --task_id=hello_world-001 --log_to_disk=DEBUG --hostname=penates.apache.org --thermos_json=/tmp/tmpMIOZ6A/task.json --sandbox=/tmp/tmp8iJqey --log_dir=/tmp/tmpT4sMgh --checkpoint_root=/tmp/tmpz3ZvsR --container_sandbox=/tmp/tmp8iJqey INFO] Forking off runner with cmdline: /usr/bin/python2.7 /tmp/tmp0fSLLb/thermos_runner.pex --task_id=hello_world-001 --log_to_disk=DEBUG --hostname=penates.apache.org --thermos_json=/tmp/tmpMIOZ6A/task.json --sandbox=/tmp/tmp8iJqey --log_dir=/tmp/tmpT4sMgh --checkpoint_root=/tmp/tmpz3ZvsR --container_sandbox=/tmp/tmp8iJqey INFO] Forking off runner with cmdline: /usr/bin/python2.7 /tmp/tmp0fSLLb/thermos_runner.pex --task_id=hello_world-001 --log_to_disk=DEBUG --hostname=penates.apache.org --thermos_json=/tmp/tmpMIOZ6A/task.json --sandbox=/tmp/tmp8iJqey --log_dir=/tmp/tmpT4sMgh --checkpoint_root=/tmp/tmpz3ZvsR --container_sandbox=/tmp/tmp8iJqey DEBUG] Waiting for task to start. DEBUG] Waiting for task to start. DEBUG] Waiting for task to start. DEBUG] - sleeping... DEBUG] - sleeping... DEBUG] - sleeping... DEBUG] - sleeping... DEBUG] - sleeping... DEBUG] - sleeping... Writing log files to disk in /tmp/tmpT4sMgh DEBUG] Task started. DEBUG] Task started. DEBUG] Task started. WARN] No health-checks defined, will use a no-op health-checker. WARN] No health-checks defined, will use a no-op health-checker. WARN] No health-checks defined, will use a no-op health-checker. INFO] Detected runner termination: pid=26479, signal=0, rc=0 INFO] Detected runner termination: pid=26479, signal=0, rc=0 INFO] Detected runner termination: pid=26479, signal=0, rc=0 DEBUG] Flipping task state from (undefined) to ACTIVE DEBUG] Flipping task state from (undefined) to ACTIVE DEBUG] Flipping task state from (undefined) to ACTIVE DEBUG] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') DEBUG] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') DEBUG] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') DEBUG] Running state machine for process=hello_world_hello_world-001/seq=0 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=0 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=0 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=1 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=1 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=1 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=2 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=2 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=2 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=3 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=3 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=3 DEBUG] Flipping task state from ACTIVE to CLEANING DEBUG] Flipping task state from ACTIVE to CLEANING DEBUG] Flipping task state from ACTIVE to CLEANING DEBUG] Flipping task state from CLEANING to FINALIZING DEBUG] Flipping task state from CLEANING to FINALIZING DEBUG] Flipping task state from CLEANING to FINALIZING DEBUG] Flipping task state from FINALIZING to SUCCESS DEBUG] Flipping task state from FINALIZING to SUCCESS DEBUG] Flipping task state from FINALIZING to SUCCESS DEBUG] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) DEBUG] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) DEBUG] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) INFO] ProvidedThermosTaskRunner reported StatusResult('Task finished.', status='TASK_FINISHED') INFO] ProvidedThermosTaskRunner reported StatusResult('Task finished.', status='TASK_FINISHED') INFO] ProvidedThermosTaskRunner reported StatusResult('Task finished.', status='TASK_FINISHED') INFO] Status manager got unhealthy status: StatusResult('Task finished.', status='TASK_FINISHED') INFO] Status manager got unhealthy status: StatusResult('Task finished.', status='TASK_FINISHED') INFO] Status manager got unhealthy status: StatusResult('Task finished.', status='TASK_FINISHED') INFO] ThermosTaskRunner is shutting down. INFO] ThermosTaskRunner is shutting down. INFO] ThermosTaskRunner is shutting down. INFO] Invoking runner.kill INFO] Invoking runner.kill INFO] Invoking runner.kill INFO] Runner is dead, skipping kill. INFO] Runner is dead, skipping kill. INFO] Runner is dead, skipping kill. INFO] Executor []: Updating hello_world-001 => FINISHED INFO] Executor []: Updating hello_world-001 => FINISHED INFO] Executor []: Updating hello_world-001 => FINISHED INFO] Executor []: Reason: Task finished. INFO] Executor []: Reason: Task finished. INFO] Executor []: Reason: Task finished. DEBUG] Flipping task state from (undefined) to ACTIVE DEBUG] Flipping task state from (undefined) to ACTIVE DEBUG] Flipping task state from (undefined) to ACTIVE DEBUG] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') DEBUG] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') DEBUG] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') DEBUG] Running state machine for process=hello_world_hello_world-001/seq=0 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=0 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=0 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=1 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=1 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=1 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=2 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=2 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=2 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=3 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=3 DEBUG] Running state machine for process=hello_world_hello_world-001/seq=3 DEBUG] Flipping task state from ACTIVE to CLEANING DEBUG] Flipping task state from ACTIVE to CLEANING DEBUG] Flipping task state from ACTIVE to CLEANING DEBUG] Flipping task state from CLEANING to FINALIZING DEBUG] Flipping task state from CLEANING to FINALIZING DEBUG] Flipping task state from CLEANING to FINALIZING DEBUG] Flipping task state from FINALIZING to SUCCESS DEBUG] Flipping task state from FINALIZING to SUCCESS DEBUG] Flipping task state from FINALIZING to SUCCESS DEBUG] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) DEBUG] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) DEBUG] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) --------------- Captured log call ---------------- executor_base.py 45 INFO I0120 10:38:16.497347 24411 executor_base.py:45] Executor [None]: TaskInfo: name: "hello_world" task_id { value: "hello_world-001" } data: "\013\000\001\000\000\000\017hello_world-001\014\000\004\014\000\031\013\000\001\000\000\000\016AuroraExecutor\013\000\002\000\000\004\314{\"priority\": 0, \"health_check_config\": {\"health_checker\": {\"http\": {\"expected_response_code\": 0, \"endpoint\": \"/health\", \"expected_response\": \"ok\"}}, \"min_consecutive_successes\": 1, \"initial_interval_secs\": 15.0, \"max_consecutive_failures\": 0, \"timeout_secs\": 1.0, \"interval_secs\": 10.0}, \"name\": \"does_not_matter\", \"service\": false, \"update_config\": {\"wait_for_batch_completion\": false, \"batch_size\": 1, \"watch_secs\": 45, \"rollback_on_failure\": true, \"max_per_shard_failures\": 0, \"max_total_failures\": 0}, \"max_task_failures\": 1, \"cron_collision_policy\": \"KILL_EXISTING\", \"enable_hooks\": false, \"instances\": 1, \"task\": {\"processes\": [{\"daemon\": false, \"name\": \"hello_world_{{thermos.task_id}}\", \"ephemeral\": false, \"max_failures\": 1, \"min_duration\": 5, \"cmdline\": \"echo hello world\", \"final\": false}], \"name\": \"hello_world\", \"finalization_wait\": 30, \"max_failures\": 1, \"max_concurrency\": 0, \"resources\": {\"gpu\": 0, \"disk\": 33554432, \"ram\": 16777216, \"cpu\": 1.0}, \"constraints\": []}, \"production\": false, \"role\": \"jenkins\", \"lifecycle\": {\"http\": {\"graceful_shutdown_endpoint\": \"/quitquitquit\", \"graceful_shutdown_wait_secs\": 5, \"port\": \"health\", \"shutdown_wait_secs\": 5, \"shutdown_endpoint\": \"/abortabortabort\"}}, \"metadata\": []}\000\014\000\034\013\000\001\000\000\000\007jenkins\013\000\002\000\000\000\003env\013\000\003\000\000\000\004name\000\014\000\035\014\000\001\000\000\000\r\000\005\013\010\000\000\000\000\010\000\006\000\000\000\000\000" executor_base.py 45 INFO I0120 10:38:16.497694 24411 executor_base.py:45] Executor [None]: launchTask got task: hello_world:hello_world-001 executor_base.py 45 INFO I0120 10:38:16.498238 24411 executor_base.py:45] Executor []: Assigned task: AssignedTask(task=TaskConfig(isService=None, contactEmail=None, taskLinks=None, container=Container(docker=None, mesos=MesosContainer(image=None, volumes=None)), mesosFetcherUris=None, partitionPolicy=None, executorConfig=ExecutorConfig(data=u'{"priority": 0, "health_check_config": {"health_checker": {"http": {"expected_response_code": 0, "endpoint": "/health", "expected_response": "ok"}}, "min_consecutive_successes": 1, "initial_interval_secs": 15.0, "max_consecutive_failures": 0, "timeout_secs": 1.0, "interval_secs": 10.0}, "name": "does_not_matter", "service": false, "update_config": {"wait_for_batch_completion": false, "batch_size": 1, "watch_secs": 45, "rollback_on_failure": true, "max_per_shard_failures": 0, "max_total_failures": 0}, "max_task_failures": 1, "cron_collision_policy": "KILL_EXISTING", "enable_hooks": false, "instances": 1, "task": {"processes": [{"daemon": false, "name": "hello_world_{{thermos.task_id}}", "ephemeral": false, "max_failures": 1, "min_duration": 5, "cmdline": "echo hello world", "final": false}], "name": "hello_world", "finalization_wait": 30, "max_failures": 1, "max_concurrency": 0, "resources": {"gpu": 0, "disk": 33554432, "ram": 16777216, "cpu": 1.0}, "constraints": []}, "production": false, "role": "jenkins", "lifecycle": {"http": {"graceful_shutdown_endpoint": "/quitquitquit", "graceful_shutdown_wait_secs": 5, "port": "health", "shutdown_wait_secs": 5, "shutdown_endpoint": "/abortabortabort"}}, "metadata": []}', name=u'AuroraExecutor'), maxTaskFailures=None, priority=None, tier=None, job=JobKey(environment=u'env', role=u'jenkins', name=u'name'), production=None, owner=None, metadata=None, resources=None, constraints=None), taskId=u'hello_world-001', instanceId=0, assignedPorts={}, slaveHost=None, slaveId=None) executor_base.py 45 INFO I0120 10:38:16.499226 24411 executor_base.py:45] Executor []: Updating hello_world-001 => STARTING executor_base.py 45 INFO I0120 10:38:16.499831 24411 executor_base.py:45] Executor []: Reason: None sandbox.py 129 DEBUG D0120 10:38:16.500787 24411 sandbox.py:129] DirectorySandbox: mkdir /tmp/tmp8iJqey sandbox.py 140 DEBUG D0120 10:38:16.501281 24411 sandbox.py:140] DirectorySandbox: chown jenkins:jenkins /tmp/tmp8iJqey sandbox.py 142 DEBUG D0120 10:38:16.501528 24411 sandbox.py:142] DirectorySandbox: chmod 700 /tmp/tmp8iJqey monitor.py 92 WARNING W0120 10:38:16.594001 24411 monitor.py:92] Could not read from checkpoint /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner thermos_task_runner.py 278 INFO I0120 10:38:16.594548 24411 thermos_task_runner.py:278] Forking off runner with cmdline: /usr/bin/python2.7 /tmp/tmp0fSLLb/thermos_runner.pex --task_id=hello_world-001 --log_to_disk=DEBUG --hostname=penates.apache.org --thermos_json=/tmp/tmpMIOZ6A/task.json --sandbox=/tmp/tmp8iJqey --log_dir=/tmp/tmpT4sMgh --checkpoint_root=/tmp/tmpz3ZvsR --container_sandbox=/tmp/tmp8iJqey thermos_task_runner.py 291 DEBUG D0120 10:38:16.601223 24411 thermos_task_runner.py:291] Waiting for task to start. thermos_task_runner.py 300 DEBUG D0120 10:38:16.601855 24411 thermos_task_runner.py:300] - sleeping... thermos_task_runner.py 300 DEBUG D0120 10:38:17.102924 24411 thermos_task_runner.py:300] - sleeping... aurora_executor.py 160 DEBUG D0120 10:38:17.612267 24411 aurora_executor.py:160] Task started. health_checker.py 390 WARNING W0120 10:38:17.692944 24411 health_checker.py:390] No health-checks defined, will use a no-op health-checker. thermos_task_runner.py 170 INFO I0120 10:38:17.694472 24411 thermos_task_runner.py:170] Detected runner termination: pid=26479, signal=0, rc=0 ckpt.py 348 DEBUG D0120 10:38:17.695306 24411 ckpt.py:348] Flipping task state from (undefined) to ACTIVE ckpt.py 328 DEBUG D0120 10:38:17.695921 24411 ckpt.py:328] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') ckpt.py 379 DEBUG D0120 10:38:17.696432 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=0 ckpt.py 379 DEBUG D0120 10:38:17.696979 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=1 ckpt.py 379 DEBUG D0120 10:38:17.697503 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=2 ckpt.py 379 DEBUG D0120 10:38:17.698055 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=3 ckpt.py 348 DEBUG D0120 10:38:17.698544 24411 ckpt.py:348] Flipping task state from ACTIVE to CLEANING ckpt.py 348 DEBUG D0120 10:38:17.698962 24411 ckpt.py:348] Flipping task state from CLEANING to FINALIZING ckpt.py 348 DEBUG D0120 10:38:17.699440 24411 ckpt.py:348] Flipping task state from FINALIZING to SUCCESS recordio.py 137 DEBUG D0120 10:38:17.699789 24411 recordio.py:137] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) status_checker.py 133 INFO I0120 10:38:17.700726 24411 status_checker.py:133] ProvidedThermosTaskRunner reported StatusResult('Task finished.', status='TASK_FINISHED') status_manager.py 61 INFO I0120 10:38:17.701132 24411 status_manager.py:61] Status manager got unhealthy status: StatusResult('Task finished.', status='TASK_FINISHED') thermos_task_runner.py 322 INFO I0120 10:38:17.702373 24411 thermos_task_runner.py:322] ThermosTaskRunner is shutting down. thermos_task_runner.py 327 INFO I0120 10:38:17.702811 24411 thermos_task_runner.py:327] Invoking runner.kill thermos_task_runner.py 219 INFO I0120 10:38:17.703178 24411 thermos_task_runner.py:219] Runner is dead, skipping kill. executor_base.py 45 INFO I0120 10:38:17.704329 24411 executor_base.py:45] Executor []: Updating hello_world-001 => FINISHED executor_base.py 45 INFO I0120 10:38:17.704688 24411 executor_base.py:45] Executor []: Reason: Task finished. ckpt.py 348 DEBUG D0120 10:38:17.712846 24411 ckpt.py:348] Flipping task state from (undefined) to ACTIVE ckpt.py 328 DEBUG D0120 10:38:17.713249 24411 ckpt.py:328] Initializing TaskRunner header to RunnerHeader(launch_time_ms=1516444697187, uid=910, task_id=u'hello_world-001', hostname=u'penates.apache.org', sandbox=u'/tmp/tmp8iJqey', log_dir=u'/tmp/tmp8iJqey/.logs', ports={}, user=u'jenkins') ckpt.py 379 DEBUG D0120 10:38:17.713608 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=0 ckpt.py 379 DEBUG D0120 10:38:17.714004 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=1 ckpt.py 379 DEBUG D0120 10:38:17.714386 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=2 ckpt.py 379 DEBUG D0120 10:38:17.714745 24411 ckpt.py:379] Running state machine for process=hello_world_hello_world-001/seq=3 ckpt.py 348 DEBUG D0120 10:38:17.715082 24411 ckpt.py:348] Flipping task state from ACTIVE to CLEANING ckpt.py 348 DEBUG D0120 10:38:17.715372 24411 ckpt.py:348] Flipping task state from CLEANING to FINALIZING ckpt.py 348 DEBUG D0120 10:38:17.715660 24411 ckpt.py:348] Flipping task state from FINALIZING to SUCCESS recordio.py 137 DEBUG D0120 10:38:17.715897 24411 recordio.py:137] /tmp/tmpz3ZvsR/checkpoints/hello_world-001/runner has no data (current offset = 607) 1 failed, 797 passed, 6 skipped in 465.61 seconds FAILURE Waiting for background workers to finish. 10:44:24 08:24 [complete] FAILURE Build step 'Execute shell' marked build as failure Recording test results