junrushao commented on code in PR #394: URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2679095465
########## docs/concepts/func_module.rst: ########## @@ -0,0 +1,584 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Function and Module +=================== + +TVM-FFI provides a unified and ABI-stable calling convention that enables +cross-language function calls between C++, Python, Rust and other languages. +Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be: + +- **Invoked with type-erased arguments** using a stable C ABI +- **Registered globally** and accessed by name across languages +- **Exported as C symbols** for efficient codegen and linking +- **Wrapped in modules** for dynamic loading and namespace isolation + +This tutorial covers everything you need to know about defining, registering, and calling +TVM-FFI functions, as well as working with modules. + +Glossary +-------- + +TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType` + A stable C calling convention where every function is represented by a single signature, + which enables type-erased, cross-language function calls. + This calling convention is used across all TVM-FFI function calls at the ABI boundary. + See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction. + +TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function` + A reference-counted function object and its managed reference, which wraps any callable, + including language-agnostic functions and lambdas (C++, Python, Rust, etc.), + member functions, external C symbols, and other callable objects, + all sharing the same calling convention. + +TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module` + A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows), + or statically linked to the current executable. + +Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func` + A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects + and their metadata (name, docs, signatures, etc.) for cross-language access. + Functions in the registry are called **global functions**. + +Common Usage +------------ + +TVM-FFI C Symbols +~~~~~~~~~~~~~~~~~ + +**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export +a function as a C symbol that follows the TVM-FFI ABI: + +.. code-block:: cpp + + static int AddTwo(int x) { return x + 2; } + + TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo) + +This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library, +and later loaded and called via :py:func:`tvm_ffi.load_module`: + +.. code-block:: python + + import tvm_ffi + + mod = tvm_ffi.load_module("path/to/library.so") + result = mod.add_two(40) # -> 42 + +**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol` +is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`. +See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow. + + +Global Functions +~~~~~~~~~~~~~~~~ + +**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to +register a function: + +.. code-block:: cpp + + #include <tvm/ffi/tvm_ffi.h> + + static int AddOne(int x) { return x + 1; } + + TVM_FFI_STATIC_INIT_BLOCK() { + namespace refl = tvm::ffi::reflection; + refl::GlobalDef() + .def("my_ext.add_one", AddOne, "Add one to the input"); + } + +:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens +during library initialization. The registered function is then accessible from +Python under the name ``my_ext.add_one``. + +In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function: + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.add_one") + def add_one(x: int) -> int: + return x + 1 + +**Retrieve a global function**. After registration, functions are accessible by name. + +In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function: + +.. code-block:: python + + import tvm_ffi + + # Get a function from the global registry + add_one = tvm_ffi.get_global_func("my_ext.add_one") + result = add_one(41) # -> 42 + +In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function: + +.. code-block:: cpp + + ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one"); + int result = func(41) // -> 42 + + +Create Functions +~~~~~~~~~~~~~~~~ + +**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped` +or :cpp:class:`tvm::ffi::TypedFunction`'s constructor. + +.. code-block:: cpp + + // Create type-erased function: add_type_erased + ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) { + return x + y; + }); + + // Create a typed function: add_typed + ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) { + return x + y; + }; + + // Converts typed function to type-erased function + ffi::Function generic = add; + +**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted +to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``: + +- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`; +- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`. + +.. code-block:: python + + import tvm_ffi + + @tvm_ffi.register_global_func("my_ext.bind") + def bind(func, x): + assert isinstance(func, tvm_ffi.Function) + return lambda *args: func(x, *args) # converted to `tvm_ffi.Function` + + def add_x_y(x, y): + return x + y + + func_bind = tvm_ffi.get_global_func("my_ext.bind") + add_y = func_bind(add_x_y, 1) # bind x = 1 + assert isinstance(add_y, tvm_ffi.Function) + print(add_y(2)) # -> 3 + + +:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`: + +.. code-block:: python + + import tvm_ffi + + def add(x, y): + return x + y + + func_add = tvm_ffi.convert(add) + print(func_add(1, 2)) + + +C ABI Calling Convention +------------------------ + +.. important:: + + This section focuses on in-depth of TVM-FFI's stable C ABI definition, + describing it in pure C for precision and clarity. + + Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI, + or to generate functions that conform to TVM-FFI ABI. + +All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature, +which provides a stable C ABI for cross-language calls. The C calling convention is defined as: + +.. code-block:: cpp + + int tvm_ffi_c_abi( + void* handle, // Resource handle + const TVMFFIAny* args, // Inputs (non-owning) + int num_args, // Number of arguments + TVMFFIAny* result // Output (owning) + ); + +.. note:: + As part of the contract, the return value ``result`` should be zero-initialized by the caller. + +The returning integer is **error code**, which indicates + +- ``0``: Success +- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised` +- ``-2``: Frontend error (e.g., Python exception) + + +Input and Output Arguments +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers. + +- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`; +- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized. + +See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`. + +Retrieve Errors in C +~~~~~~~~~~~~~~~~~~~~ + +When a TVM-FFI function returns a non-zero code, it indicates an error occurred, +and the error object is stored in the TLS slot. This section shows how to retrieve +the error object and print the error message and backtrace. + +.. hint:: + Compiler code generation may use similar patterns to call any function with TVM-FFI ABI. + + +**Print an Error**. The example code below shows how to print an error message and backtrace. + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + static void PrintError(TVMFFIObject* err) { + TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject)); + fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data); + if (cell->backtrace.size) { + fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data); + } + } + +The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure, +which contains the error kind, message, and backtrace, which can be accessed +by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics. + +.. note:: + + An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload + as defined below: + + .. code-block:: cpp + + typedef struct { + TVMFFIByteArray kind; // Error type (e.g., "ValueError") + TVMFFIByteArray message; // Error message + TVMFFIByteArray backtrace; // Stack trace (most-recent call first) + void (*update_backtrace)(...); // Hook to append/replace backtrace + } TVMFFIErrorCell; + + +**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot, +and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`. + +.. code-block:: cpp + + void HandleReturnCode(int rc) { + TVMFFIObject* err = NULL; + if (rc == 0) { + // Success + } + else if (rc == -1) { + // Move the raised error from TLS (clears TLS slot) + TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object + if (err != NULL) { + PrintError(err); // print the error + TVMFFIObjectDecRef(err); // Release the error object + } + } + else if (rc == -2) { + // Frontend (e.g., Python) already has an exception set. + // Do not fetch from TLS; consult the frontend's error mechanism. + } + } + +This method transfers ownership of the error object to the caller and clears the TLS slot, +and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done +to avoid memory leaks. + +Raise Errors in C +~~~~~~~~~~~~~~~~~ + +As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred, +and the error object is stored in the TLS slot. The error object may contain abitrary +user-defined information, for example, error message, backtrace, or Python frame-local variables. + +.. hint:: + Compiler code generation may use similar patterns to raise errors in generated code. + +The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`: + +.. code-block:: cpp + + #include <tvm/ffi/c_api.h> + + int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args, + int32_t num_args, TVMFFIAny* result) { + // Validate inputs + if (num_args < 2) { + TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments"); + return -1; + } + // ... kernel implementation ... + return 0; + } + +Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate +multiple parts of the error message into a single string. + +**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries +the error details: + +.. code-block:: cpp + + int outer_function(...) { + int err_code = 0; + + err_code = inner_function(...); + if (err_code != 0) goto RAII; // Propagate error; TLS has the details + + RAII: + // clean up owned resources + return err_code; + } + +Handling Non-C++ Errors +~~~~~~~~~~~~~~~~~~~~~~~ + +TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path. Review Comment: Updated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
