pfk-beta opened a new issue #10240:
URL: https://github.com/apache/tvm/issues/10240
Hello,
I'm using heavily RPC tracker and server with TVM. I encounter many times
issue, when tuning task cause device reboot or rpc_android_app restart. After
that, whole tuning process need to be restarted. Tuning process waits for
something - when I turn RPC_APP again, it breaks.
Error from tuning process:
```
[Task 4/16] Current/Best: 0.29/ 0.96 GFLOPS | Progress: (10/10) |
89.11 s Done.
[Task 5/16] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/10) | 0.00
sTraceback (most recent call last):
File "tests/android_auto.py", line 180, in <module>
tune_and_evaluate(tuning_option)
File "tests/android_auto.py", line 142, in tune_and_evaluate
tune_tasks(tasks, **tuning_opt)
File "tests/android_auto.py", line 120, in tune_tasks
autotvm.callback.log_to_file(tmp_log_file),
File "/workspace/python/tvm/autotvm/tuner/xgboost_tuner.py", line 105, in
tune
super(XGBTuner, self).tune(*args, **kwargs)
File "/workspace/python/tvm/autotvm/tuner/tuner.py", line 112, in tune
measure_batch = create_measure_batch(self.task, measure_option)
File "/workspace/python/tvm/autotvm/measure/measure.py", line 282, in
create_measure_batch
attach_objects = runner.set_task(task)
File "/workspace/python/tvm/autotvm/measure/measure_methods.py", line 326,
in set_task
"set_task - Cannot get remote devices from the tracker. "
RuntimeError: set_task - Cannot get remote devices from the tracker. Please
check the status of tracker by 'python -m tvm.exec.query_rpc_tracker --port
[THE PORT YOU USE]' and make sure you have free devices on the queue status.
```
Ouptput from query_rpc_tracker:
```
Tracker address 0.0.0.0:9190
Server List
----------------------------
server-address key
----------------------------
----------------------------
Queue Status
-------------------------------
key total free pending
-------------------------------
android 0 0 1
-------------------------------
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
