I seem to have issues running the end to end tests on master. Seems to get
stuck in a ZooKeeper related loop detecting no active schedulers:
...
BUILD SUCCESSFUL
Total time: 19.711 secs
I0501 22:12:27.352841 4901 leveldb.cpp:176] Opened db in 1.990798ms
I0501 22:12:27.354225 4901 leveldb.cpp:183] Compacted db in 1.04968ms
I0501 22:12:27.354480 4901 leveldb.cpp:198] Created db iterator in 16569ns
I0501 22:12:27.354573 4901 leveldb.cpp:204] Seeked to beginning of db in
80366ns
I0501 22:12:27.354867 4901 leveldb.cpp:273] Iterated through 10 keys in
the db in 112016ns
I0501 22:12:27.355093 4901 replica.cpp:744] Replica recovered with log
positions 754 -> 762 with 0 holes and 0 unlearned
The log is not empty
Replicated log initialization failed with code 1 (likely already
initialized).
aurora-scheduler stop/waiting
aurora-scheduler start/running, process 4917
+ test_version
++ aurora --version
++ cat /vagrant/.auroraversion
+ [[ 0.7.1-SNAPSHOT = 0.7.1-SNAPSHOT ]]
+ test_http_example devcluster vagrant test http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
+ local _cluster=devcluster _role=vagrant _env=test _job=http_example
+ local
_base_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
_updated_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
+ local _jobkey=devcluster/vagrant/test/http_example
+ test_config
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
devcluster/vagrant/test/http_example
+ local
_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
_jobkey=devcluster/vagrant/test/http_example
++ tr -dc '[[:print:]]'
++ aurora config list
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ joblist='jobs=[devcluster/vagrant/test/http_example]'
+ [[ jobs=[devcluster/vagrant/test/http_example] =
\j\o\b\s\=\[\d\e\v\c\l\u\s\t\e\r\/\v\a\g\r\a\n\t\/\t\e\s\t\/\h\t\t\p\_\e\x\a\m\p\l\e\]
]]
+ test_inspect devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ local _jobkey=devcluster/vagrant/test/http_example
_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ aurora job inspect devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
Job level information
name: 'http_example'
role: 'vagrant'
contact: 'vagrant@localhost'
cluster: 'devcluster'
instances: '2'
constraints:
'host': 'limit:2'
service: True
production: False
Task level information
name: 'http_example'
constraints:
stage_server < run_server
Process 'stage_server':
cmdline:
cp /vagrant/src/test/sh/org/apache/aurora/e2e/http_example.py .
Process 'run_server':
cmdline:
python http_example.py {{thermos.ports[http]}}
+ test_create devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ local _jobkey=devcluster/vagrant/test/http_example
_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ aurora job create devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
INFO] Creating job http_example
INFO] Connecting to 192.168.33.7:2181
INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
INFO] Zookeeper connection established, state: CONNECTED
INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
watcher=<function get_watch at 0x7f5a8fc28578>)
INFO] Received response(xid=1): []
INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
watcher=None)
INFO] Received response(xid=2): []
WARN] Could not connect to scheduler: No schedulers detected in devcluster!
INFO] Connecting to 192.168.33.7:2181
INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
INFO] Zookeeper connection established, state: CONNECTED
INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
watcher=<function get_watch at 0x7f5a8fc28cf8>)
INFO] Received response(xid=1): []
INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
watcher=None)
INFO] Received response(xid=2): []
WARN] Could not connect to scheduler: No schedulers detected in devcluster!
INFO] Connecting to 192.168.33.7:2181
INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
INFO] Zookeeper connection established, state: CONNECTED
INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
watcher=<function get_watch at 0x7f5a8fc8c398>)
INFO] Received response(xid=1): []
INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
watcher=None)
INFO] Received response(xid=2): []
WARN] Could not connect to scheduler: No schedulers detected in devcluster!
INFO] Connecting to 192.168.33.7:2181
INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
...
This last part continues on forever as far as I can tell.
Is this a known issue?
Ben
