pfk-beta opened a new issue, #11058:
URL: https://github.com/apache/tvm/issues/11058
Hello,
I think that I have correct rpc connection between my computer and android
device:
1. I have compiled tvm_rpc executable using Docker.android_demo with
modifiication. I have put there headers for OpenCL, and compiled binary to use
them.
2. I have put executable to the device, and `libc++_shared.so`
`libtvm_runtime.so` libraries, as well.
3. When I run command `adb -s HQUCI76PJJ7TFYLZ shell
"LD_LIBRARY_PATH=/data/local/tmp /data/local/tmp/tvm_rpc server
--host=192.168.1.122 --port=5000 --tracker=192.168.1.135:9190 --key=prague
--work-dir=/data/local/tmp/"`, it looks like correct working. I see following
info:
```
[14:28:48] /workspace/apps/cpp_rpc/main.cc:96: host = 192.168.1.122
[14:28:48] /workspace/apps/cpp_rpc/main.cc:97: port = 5000
[14:28:48] /workspace/apps/cpp_rpc/main.cc:98: port_end = 9099
[14:28:48] /workspace/apps/cpp_rpc/main.cc:99: tracker =
('192.168.1.135', 9190)
[14:28:48] /workspace/apps/cpp_rpc/main.cc:100: key = prague
[14:28:48] /workspace/apps/cpp_rpc/main.cc:101: custom_addr =
[14:28:48] /workspace/apps/cpp_rpc/main.cc:102: work_dir =
/data/local/tmp/
[14:28:48] /workspace/apps/cpp_rpc/main.cc:103: silent = False
[14:28:48] /workspace/apps/cpp_rpc/main.cc:264: Starting CPP Server, Press
Ctrl+C to stop.
[14:28:48] /workspace/apps/cpp_rpc/rpc_server.cc:130: bind to
192.168.1.122:5000
[14:28:48] /workspace/apps/cpp_rpc/rpc_tracker_client.h:201: Tracker
connecting to 192.168.1.135:9190
```
4. I have compiled runtime on my computer, with RPC and LLVM on. When I run
rpc_tracker, and query rpc_tracker I see following infomration (I think it is
working correctly):
```
Tracker address 192.168.1.135:9190
Server List
----------------------------
server-address key
----------------------------
192.168.1.210:5000 server:prague
192.168.1.122:5000 server:prague
```
```
python3 -m tvm.exec.rpc_tracker --port 9190
INFO:RPCTracker:bind to 0.0.0.0:9190
```
5. When I run tuning script, `tvm_rpc` gives me little message `[14:23:59]
/workspace/apps/cpp_rpc/rpc_server.cc:300: Connection success
192.168.1.135:60674`. Then I get stacktrace from tuning script:
```
File
"/home/piotr/projects/odai-meta/odai-tvm/docker/tcl/tcl_scripts/tune.py", line
152, in <module>
main(args)
File
"/home/piotr/projects/odai-meta/odai-tvm/docker/tcl/tcl_scripts/tune.py", line
99, in main
tune_model(args, mod, params, target)
File
"/home/piotr/projects/odai-meta/odai-tvm/docker/tcl/tcl_scripts/tune.py", line
77, in tune_model
task_tuner.tune(
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/autotvm/tuner/xgboost_tuner.py",
line 105, in tune
super(XGBTuner, self).tune(*args, **kwargs)
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/autotvm/tuner/tuner.py",
line 112, in tune
measure_batch = create_measure_batch(self.task, measure_option)
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/autotvm/measure/measure.py",
line 282, in create_measure_batch
attach_objects = runner.set_task(task)
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/autotvm/measure/measure_methods.py",
line 291, in set_task
raise RuntimeError(
RuntimeError: Cannot get remote devices from the tracker. Please check the
status of tracker by 'python -m tvm.exec.query_rpc_tracker --port [THE PORT YOU
USE]' and make sure you have free devices on the queue status.
```
(But, in another terminal I'm runing query in `watch` command, so it is
working correctly).
6. After one minute I see in `tvm_rpc` following message:
```
[14:24:59] /workspace/apps/cpp_rpc/rpc_server.cc:198: Child pid=12547 killed
(timeout = 60), Process status = 15
[14:24:59] /workspace/apps/cpp_rpc/rpc_server.cc:229: Socket Connection
Closed
```
and tuning script is exiting with following stacktrace:
```
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/autotvm/measure/measure_methods.py",
line 769, in _check
while not dev.exist: # wait until we get an available device
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/_ffi/runtime_ctypes.py",
line 262, in exist
return self._GetDeviceAttr(self.device_type, self.device_id, 0) != 0
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/_ffi/runtime_ctypes.py",
line 246, in _GetDeviceAttr
return tvm.runtime._ffi_api.GetDeviceAttr(device_type, device_id,
attr_id)
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/python/tvm/_ffi/_ctypes/packed_func.py",
line 237, in __call__
raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
4: TVMFuncCall
3: std::_Function_handler<void (tvm::runtime::TVMArgs,
tvm::runtime::TVMRetValue*), __mk_TVM1::{lambda(tvm::runtime::TVMArgs,
tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&,
tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
2: tvm::runtime::RPCDeviceAPI::GetAttr(DLDevice,
tvm::runtime::DeviceAttrKind, tvm::runtime::TVMRetValue*)
1: non-virtual thunk to tvm::runtime::RPCClientSession::GetAttr(DLDevice,
tvm::runtime::DeviceAttrKind, tvm::runtime::TVMRetValue*)
0: std::_Function_handler<void (tvm::runtime::TVMArgs,
tvm::runtime::TVMRetValue*),
tvm::runtime::RPCEndpoint::Init()::{lambda(tvm::runtime::TVMArgs,
tvm::runtime::TVMRetValue*)#2}>::_M_invoke(std::_Any_data const&,
tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)
File
"/home/piotr/projects/odai-meta/odai-tvm/tvm/src/runtime/rpc/rpc_endpoint.cc",
line 681
TVMError:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: (code == RPCCode::kReturn) is false: code=1
Done.
```
7. Both, in building tvm_rpc, and in runing tuning script I'm using
sourcecode from `v0.9.dev0` git tag, is it ok?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
