echuraev commented on a change in pull request #7876: URL: https://github.com/apache/tvm/pull/7876#discussion_r711989713
########## File path: apps/ios_rpc/README.md ########## @@ -114,19 +57,198 @@ cmake .. -DCMAKE_OSX_ARCHITECTURES=arm64 -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0 -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON - -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX # insert your Team ID -DUSE_IOS_RPC=ON # to enable build iOS RPC application from TVM project tree -cmake --build . --target custom_dso_loader ios_rpc # Will use custom DSO loader by default -# Resulting iOS RPC app bundle will be placed in: -# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app + -DUSE_METAL=ON # to enable Metal runtime + +cmake --build . --target custom_dso_loader tvm_runtime +``` + +### Building iOS TVM RPC application +Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata: +```shell +python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder" +``` +You can get value of your `team_id` in the following ways: +- **You have registered Apple Developer Profile**. In this case you developer + Team ID available at https://developer.apple.com/account/#/membership +- You are using your local developer profile. In this case run the command above + with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using + XCode, click on the project name (`tvmrpc`) on the left panel. Then select + target `tvmrpc`. At the bottom of this panel go to `Signing & Capabilities` + tab and in the field `Team` select your local developer profile + (`Your Name (Personal Team)`). + + On the first run of the application you may see message `Could not launch + "tvmrpc"` in the XCode and message `Untrusted Developer` on your device. In + this case it will be necessary to check the certificate. Open + `Settings -> General -> Device Management -> Apple Development: <your_email> + -> Trust "Apple Development: <your_email>"` and click `Trust`. After than you + should rerun your application in the XCode. + +After this step, open `tvmrpc.xcodeproj` by using XCode, build the App and +install the App on the phone. + +## Workflow +Due to security restriction of iOS10. We cannot upload dynamic libraries to the +App and load it from sandbox. Instead, we need to build a list of libraries, +pack them into the app bundle, launch the RPC server and connect to test the +bundled libraries. We use one approach to workaround this limitation, for more +details please take a look into section +[Building TVM runtime and custom DSO loader integration](#building-tvm-runtime-and-custom-DSO-loader-plugin). + +The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) and +[tests/ios_rpc_mobilenet.py](tests/ios_rpc_mobilenet.py) are good templates for +demonstrating the workflow. + +We have three different modes for iOS RPC server: +- [Pure RPC](#pure-rpc): In this mode RPC server open port on the device and listening. Then + client connects to the server directly without any mediators. +- [iOS RPC application with Proxy](#ios-rpc-app-with-proxy): RPC server and RPC client communicates through + `rpc_proxy`. The RPC server on iOS device notify `rpc_proxy` which was run on + host machine about itself and wait for incoming connections. Communications + between client and server works through `rpc_proxy`. +- [iOS RPC application with Tracker](#ios-rpc-app-with-tracker): RPC server registered in the `rpc_tracker` + and client connects to the RPC server through `rpc_tracker`. + +### Pure RPC +Start RPC server on your iOS device: +- Enable verbose output. +- Push on the `Connect` button. + +After that you supposed to see something like this in the app on the device: +``` +IP: <device_ip> +Port: <rpc_server_port> ``` -To enable using of Custom DSO Plugin during xcode build outsde of Cmake you should specify two additional variables. -You can do it manually inside Xcode IDE or via command line args for `xcodebuild`. Make sure that `custom_dso_loader` -target from previous step is already built. -* TVM_BUILD_DIR=path-to-tvm-ios-build-dir -* USE_CUSTOM_DSO_LOADER=1 +Printed `IP` is the IP address of your device and `PORT` is the number of port +which was open for RPC connection. Next you should use them for connect your RPC +client to the server. -iOS RPC application with enabled custom DSO loader is able to process modules passed via regular -`remote.upload("my_module.dylib")` mechanics. For example take a look inside `test_rpc_module_with_upload` test case -of file [ios_rpc_test.py](tests/ios_rpc_test.py). +Let's check that direct RPC connection works and we can upload a library with +model and execute it on the device. For this purpose we will use +[ios_rpc_test.py](tests/ios_rpc_test.py). Run it: +```shell +python3 tests/ios_rpc_test.py --host <device_ip> --port <rpc_server_port> --mode "pure_server" +``` +This will compile TVM IR to shared libraries (CPU and Metal) and run vector +addition on your iOS device. You are supposed to see something like this: +``` +Metal: 0.000338692 secs/op +CPU: 0.000219308 secs/op +``` + +### iOS RPC App with proxy +Start the RPC proxy by running in a terminal: +```shell +python3 -m tvm.exec.rpc_proxy --host 0.0.0.0 --port 9090 +``` + +On success, you should see something like this: +``` +INFO:root:RPCProxy: client port bind to 0.0.0.0:9090 +INFO:root:RPCProxy: Websock port bind to 8888 +``` +Connect your iOS device to the RPC proxy via the iOS TVM RPC application. Set +the `Address` and `Port` fields to the address and port of the RPC tracker +respectively. Select mode `Proxy` and push `Connect` button. In success the +text on the button will be changed to `Disconnect` and `Disconnected` in the top +of the screen will be changed to `Connected`. +On RPC proxy side you can see the next message in a log: +``` +INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone +``` +Then we can check that RPC connection works and we can upload a library with +model and execute it on the target device. For this purpose we will use +[ios_rpc_test.py](tests/ios_rpc_test.py). Run it: +```shell +python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9090 --mode "proxy" +``` +The output should be the same as it was in previous section. + +### iOS RPC App with tracker +First start an RPC tracker using +```shell +python3 -m tvm.exec.rpc_tracker --host 0.0.0.0 --port 9190 +``` +On success, you should see something like this: +``` +INFO:RPCTracker:bind to 0.0.0.0:9190 +``` +Connect your iOS device to the RPC tracker via the iOS TVM RPC applcation. Set +the `Address` and `Port` fields to the address and port of the RPC tracker +respectively. Select mode `Tracker` and push `Connect` button. In success the +text on the button will be changed to `Disconnect` and `Disconnected` in the top +of the screen will be changed to `Connected`. On the host side you can check the +connect by the following command: +```shell +python3 -m tvm.exec.query_rpc_tracker --port 9190 +``` +You are supposed to see something like this: +``` +Tracker address 127.0.0.1:9190 + +Server List +---------------------------- +server-address key +---------------------------- +192.168.1.57:9190 server:iphone +---------------------------- + +Queue Status +------------------------------ +key total free pending +------------------------------ +iphone 1 1 0 +------------------------------ +``` + +Then we can check that RPC connection works and we can upload a library with +model and execute it on the target device. For this purpose we will use +[ios_rpc_test.py](tests/ios_rpc_test.py). Run it: +```shell +python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9190 --mode "tracker" +``` +The output will be the same as in section +[Pure RPC](#pure-rpc). + +## Communication without Wi-Fi and speed up in case of slow Wi-Fi +Connection to the RPC server through `usbmux` can be used then you have slow, +unstable or don't have any Wi-Fi connection. `usbmux` is used for binding local +TCP port to port on the device and transfer packages between these ports by USB +cable. + +First of all you should install `usbmux` to your system. You can do it with +brew: +```shell +brew install usbmuxd +``` +After that you can use `iproxy` program for binding ports. You can use it for +all described workflows. Let's take a look how it works for [Pure RPC](#pure-rpc). Review comment: It should work for all other modes also. I say about it in the sentence `You can use it for all described workflows.`. Just for example, I wrote about Pure RPC. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
