yaoyaoding commented on code in PR #283: URL: https://github.com/apache/tvm-ffi/pull/283#discussion_r2558643071
########## examples/cubin_launcher/src/lib_embedded.cc: ########## @@ -0,0 +1,130 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/*! + * \file examples/cubin_launcher/src/lib_embedded.cc + * \brief TVM-FFI library with embedded CUBIN kernels. + * + * This library exports TVM-FFI functions to launch CUDA kernels from + * embedded CUBIN data. + */ + +#include <tvm/ffi/container/tensor.h> +#include <tvm/ffi/error.h> +#include <tvm/ffi/extra/c_env_api.h> +#include <tvm/ffi/extra/cubin_launcher.h> +#include <tvm/ffi/function.h> + +#include <cstdint> +#include <memory> + +// External symbols for embedded CUBIN data (linked via objcopy) +extern "C" const char __cubin_data[]; +extern "C" const char __cubin_data_end[]; Review Comment: I've added a python script to embed cubin to an object file: ``` usage: python -m tvm_ffi.utils.embed_cubin [-h] --output-obj PATH --input-obj PATH --cubin PATH --name NAME [-v] Embed CUBIN data into existing object files that use TVM_FFI_EMBED_CUBIN macro options: -h, --help show this help message and exit --output-obj PATH Output object file path (e.g., new.o) --input-obj PATH Input object file path containing TVM_FFI_EMBED_CUBIN usage (e.g., old.o) --cubin PATH Input CUBIN file path (e.g., kernel.cubin) --name NAME Name used in TVM_FFI_EMBED_CUBIN macro (e.g., my_kernels) -v, --verbose Print detailed command output Examples: # Basic usage python -m tvm_ffi.utils.embed_cubin \ --output-obj new.o \ --input-obj old.o \ --cubin kernel.cubin \ --name my_kernels # With verbose output python -m tvm_ffi.utils.embed_cubin \ --output-obj new.o \ --input-obj old.o \ --cubin kernel.cubin \ --name my_kernels \ --verbose Workflow: 1. Compile C++ code that uses TVM_FFI_EMBED_CUBIN to create old.o 2. Compile CUDA kernel to CUBIN (e.g., using nvcc or NVRTC) 3. Use this tool to merge them into new.o 4. Link new.o into your final shared library Usage in C++ code (source compiled to old.o): TVM_FFI_EMBED_CUBIN(my_kernels); auto kernel = TVM_FFI_EMBED_CUBIN_GET_KERNEL(my_kernels, "kernel_name"); Requirements: - GNU binutils (ld and objcopy) must be available in PATH - Linux/Unix platform (Windows uses different embedding mechanisms) ``` For cmake, we have two utility functions: ```cmake tvm_ffi_generate_cubin( OUTPUT <output_cubin_file> SOURCE <cuda_source_file> [ARCH <architecture>] [OPTIONS <extra_nvcc_options>...] [DEPENDS <additional_dependencies>...] ) tvm_ffi_embed_cubin( OUTPUT <output_object_file> SOURCE <source_file> CUBIN <cubin_file> NAME <symbol_name> [DEPENDS <additional_dependencies>...] ) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
