gemini-code-assist[bot] commented on code in PR #164:
URL: https://github.com/apache/tvm-ffi/pull/164#discussion_r2440520187
##########
docs/guides/packaging.md:
##########
@@ -14,21 +14,198 @@
<!--- KIND, either express or implied. See the License for the -->
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->
-# Packaging
+# Python Binding and Packaging
-This guide explains how to package a tvm-ffi-based library into a Python
ABI-agnostic wheel.
-It demonstrates both source-level builds (for cross-compilation) and builds
based on pre-shipped shared libraries.
+This guide explains how to leverage tvm-ffi to expose C++ functions into
Python and package them into a wheel.
At a high level, packaging with tvm-ffi offers several benefits:
-- **ABI-agnostic wheels**: Works across different Python versions with minimal
dependency.
-- **Universally deployable**: Build once with tvm-ffi and ship to different
environments, including Python and non-Python environments.
+- **Ship one wheel** that can be used across Python versions, including
free-threaded Python.
+- **Cross language** abd accessed the binded functions any other languages,
such as Rust.
+- **ML Systems Interop** to interact with DSLs, libraries, and other
frameworks that interact with the ABI.
-While this guide shows how to build a wheel package, the resulting
`my_ffi_extension.so` is agnostic
-to Python, comes with minimal dependencies, and can be used in other
deployment scenarios.
-## Build and Run the Example
+## Directly using Exported Library
-Let's start by building and running the example.
+If you just need to expose a simple set of functions,
+you can declare an exported symbol in C++:
+
+```c++
+// Compiles to mylib.so
+#include <tvm/ffi/function.h>
+
+int add_one(int x) {
+ return x + 1;
+}
+
+TVM_FFI_DLL_EXPORT_TYPED_FUNC(add_one, add_one)
+```
+
+You then load the exported function in your Python project via
{py:func}`tvm_ffi.load_module`.
+
+```python
+# In your __init__.py
+import tvm_ffi
+
+_LIB = tvm_ffi.load_module("/path/to/mlib.so")
+
+def add_one(x):
+ """Expose mylib.add_one"""
+ return _LIB.add_one(x)
+```
+
+This approach is like using {py:mod}`ctypes` load and run DLLs, except we have
more powerful features:
+
+- We can pass in and return a richer set of data types such as
{py:class}`tvm_ffi.Tensor` and strings.
+- {py:class}`tvm_ffi.Function` enables natural callback into Python lambdas or
other languages.
+- Exceptions are propagated naturally across language boundaries.
+
+## Pybind11 and Nanobind style Usage
+
+For advanced use cases where users may wish to register global functions or
custom object types.
+We also provide a pybind11/nanobind style API to register functions and custom
objects
+
+```c++
+#include <tvm/ffi/error.h>
+#include <tvm/ffi/reflection/registry.h>
+
+namespace my_ffi_extension {
+
+namespace ffi = tvm::ffi;
+
+/*!
+ * \brief Example of a custom object that is exposed to the FFI library
+ */
+class IntPairObj : public ffi::Object {
+ public:
+ int64_t a;
+ int64_t b;
+
+ IntPairObj() = default;
+ IntPairObj(int64_t a, int64_t b) : a(a), b(b) {}
+
+ int64_t GetFirst() const { return this->a; }
+
+ // Required: declare type information
+ TVM_FFI_DECLARE_OBJECT_INFO_FINAL("my_ffi_extension.IntPair", IntPairObj,
ffi::Object);
+};
+
+/*!
+ * \brief Defines an explicit reference to IntPairObj
+ *
+ * A reference wrapper serves as a reference-counted pointer to the object.
+ * You can use obj->field to access the fields of the object.
+ */
+class IntPair : public tvm::ffi::ObjectRef {
+ public:
+ // Constructor
+ explicit IntPair(int64_t a, int64_t b) {
+ data_ = tvm::ffi::make_object<IntPairObj>(a, b);
+ }
+
+ // Required: define object reference methods
+ TVM_FFI_DEFINE_OBJECT_REF_METHODS_NULLABLE(IntPair, tvm::ffi::ObjectRef,
IntPairObj);
+};
+
+void RaiseError(ffi::String msg) { TVM_FFI_THROW(RuntimeError) << msg; }
+
+TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ffi_extension.raise_error", RaiseError);
+ // register object definition
+ refl::ObjectDef<IntPairObj>()
+ .def(refl::init<int64_t, int64_t>())
+ // Example static method that returns the first element of the pair
Review Comment:
The comment is incorrect. The associated method `static_get_second` returns
the second element of the pair (`pair->b`), not the first.
```suggestion
// Example static method that returns the second element of the pair
```
##########
docs/guides/packaging.md:
##########
@@ -14,21 +14,198 @@
<!--- KIND, either express or implied. See the License for the -->
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->
-# Packaging
+# Python Binding and Packaging
-This guide explains how to package a tvm-ffi-based library into a Python
ABI-agnostic wheel.
-It demonstrates both source-level builds (for cross-compilation) and builds
based on pre-shipped shared libraries.
+This guide explains how to leverage tvm-ffi to expose C++ functions into
Python and package them into a wheel.
At a high level, packaging with tvm-ffi offers several benefits:
-- **ABI-agnostic wheels**: Works across different Python versions with minimal
dependency.
-- **Universally deployable**: Build once with tvm-ffi and ship to different
environments, including Python and non-Python environments.
+- **Ship one wheel** that can be used across Python versions, including
free-threaded Python.
+- **Cross language** abd accessed the binded functions any other languages,
such as Rust.
+- **ML Systems Interop** to interact with DSLs, libraries, and other
frameworks that interact with the ABI.
-While this guide shows how to build a wheel package, the resulting
`my_ffi_extension.so` is agnostic
-to Python, comes with minimal dependencies, and can be used in other
deployment scenarios.
-## Build and Run the Example
+## Directly using Exported Library
-Let's start by building and running the example.
+If you just need to expose a simple set of functions,
+you can declare an exported symbol in C++:
+
+```c++
+// Compiles to mylib.so
+#include <tvm/ffi/function.h>
+
+int add_one(int x) {
+ return x + 1;
+}
+
+TVM_FFI_DLL_EXPORT_TYPED_FUNC(add_one, add_one)
+```
+
+You then load the exported function in your Python project via
{py:func}`tvm_ffi.load_module`.
+
+```python
+# In your __init__.py
+import tvm_ffi
+
+_LIB = tvm_ffi.load_module("/path/to/mlib.so")
+
+def add_one(x):
+ """Expose mylib.add_one"""
+ return _LIB.add_one(x)
+```
+
+This approach is like using {py:mod}`ctypes` load and run DLLs, except we have
more powerful features:
+
+- We can pass in and return a richer set of data types such as
{py:class}`tvm_ffi.Tensor` and strings.
+- {py:class}`tvm_ffi.Function` enables natural callback into Python lambdas or
other languages.
+- Exceptions are propagated naturally across language boundaries.
+
+## Pybind11 and Nanobind style Usage
+
+For advanced use cases where users may wish to register global functions or
custom object types.
+We also provide a pybind11/nanobind style API to register functions and custom
objects
+
+```c++
+#include <tvm/ffi/error.h>
+#include <tvm/ffi/reflection/registry.h>
+
+namespace my_ffi_extension {
+
+namespace ffi = tvm::ffi;
+
+/*!
+ * \brief Example of a custom object that is exposed to the FFI library
+ */
+class IntPairObj : public ffi::Object {
+ public:
+ int64_t a;
+ int64_t b;
+
+ IntPairObj() = default;
+ IntPairObj(int64_t a, int64_t b) : a(a), b(b) {}
+
+ int64_t GetFirst() const { return this->a; }
+
+ // Required: declare type information
+ TVM_FFI_DECLARE_OBJECT_INFO_FINAL("my_ffi_extension.IntPair", IntPairObj,
ffi::Object);
+};
+
+/*!
+ * \brief Defines an explicit reference to IntPairObj
+ *
+ * A reference wrapper serves as a reference-counted pointer to the object.
+ * You can use obj->field to access the fields of the object.
+ */
+class IntPair : public tvm::ffi::ObjectRef {
+ public:
+ // Constructor
+ explicit IntPair(int64_t a, int64_t b) {
+ data_ = tvm::ffi::make_object<IntPairObj>(a, b);
+ }
+
+ // Required: define object reference methods
+ TVM_FFI_DEFINE_OBJECT_REF_METHODS_NULLABLE(IntPair, tvm::ffi::ObjectRef,
IntPairObj);
+};
+
+void RaiseError(ffi::String msg) { TVM_FFI_THROW(RuntimeError) << msg; }
+
+TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ffi_extension.raise_error", RaiseError);
+ // register object definition
+ refl::ObjectDef<IntPairObj>()
+ .def(refl::init<int64_t, int64_t>())
+ // Example static method that returns the first element of the pair
+ .def_static("static_get_second", [](IntPair pair) -> int64_t { return
pair->b; })
+ // Example to bind an instance method
+ .def("get_first", &IntPairObj::GetFirst)
+ .def_ro("a", &IntPairObj::a)
+ .def_ro("b", &IntPairObj::b);
+}
+} // namespace my_ffi_extension
+```
+
+Then these functions and objects can be accessed from Python as long as the
library is loaded.
+You can use {py:func}`tvm_ffi.load_module` or simply use
{py:class}`ctypes.CDLL`. Then you can access
+the function through {py:func}`tvm_ffi.get_global_func` or
{py:func}`tvm_ffi.init_ffi_api`.
+We also allow direct exposure of object via {py:func}`tvm_ffi.register_object`.
+
+```python
+# __init__.py
+import tvm_ffi
+
+def raise_error(msg: str):
+ """Wrap raise error function."""
+ # Usually we reorganize these functions into a _ffi_api.py and load once
+ func = tvm_ffi.get_global_func("my_ffi_extension.raise_error")
+ func(msg)
+
+
+@tvm_ffi.register_object("my_ffi_extension.IntPair")
+class IntPair(tvm_ffi.Object):
+ """IntPair object."""
+
+ def __init__(self, a: int, b: int) -> None:
+ """Construct the object."""
+ # __ffi_init__ call into the refl::init<> registered
+ # in the static initialization block of the extension library
+ self.__ffi_init__(a, b)
+
+
+def run_example():
+ pair = IntPair(1, 2)
+ # prints 1
+ print(pair.get_first())
+ # prints 2
+ print(IntPair.static_get_second(pair))
+ # Raises a RuntimeError("error happens")
+ raise_error("error happens")
+```
+
+### Relations to Existing Solutions
+
+Most current binding systems focus on creating one-to-one bindings
+that take a source language and bind to an existing target language runtime
and ABI.
+We deliberately take a more decoupled approach here:
+
+- Build stable, minimal ABI convention that is agnostic to the target language.
+- Create bindings to connect the source and target language to the ABI.
+
+The focus of this project is the ABI itself which we believe can help the
overall ecosystem.
+We also anticipate there are possibilities for existing binding generators to
also target the tvm-ffi ABI.
+
+**Design philosophy**. We have the following design philosophies focusing on
ML systems.
+
+- FFI and cross-language interop should be first-class citizens in ML systems
rather than an add-on.
+- Enable multi-environment support in both source and target languages.
+- The same ABI should be minimal and targetable by DSL compilers.
+
+Of course, there is always a tradeoff. It is by design impossible to support
arbitrary advanced language features
+in the target language, as different programming languages have their own
design considerations.
+We do believe it is possible to build a universal, effective, and minimal ABI
for machine learning
+system use cases. Based on the above design philosophies, we focus our
cross-language
+interaction interface through the FFI ABI for machine learning systems.
+
+So if you are building projects related to machine learning compilers,
runtimes, libraries,
+frameworks, DSLs, or generally scientific computing, we definitely encourage
you to use it.
+The extension mechanism can likely support features in other domains as well
and we welcome you to
+try it out as well.
+
+### Mix with Existing Solutions
+
+Because the global registry mechanism only relies on the code being linked,
+you can also partially use tvm-ffi-based registration together with
pybind11/nanobind in your project.
+Just add the related code, link to `libtvm_ffi`m and make sure you `import
tvm_ffi` before importing
Review Comment:
There's a typo here: `libtvm_ffi`m should be `libtvm_ffi`. Note that
markdown formatting (backticks) has been removed from the suggestion to comply
with output requirements; please re-add it when applying the change.
```suggestion
Just add the related code, link to libtvm_ffi and make sure you import
tvm_ffi before importing
```
##########
examples/packaging/src/extension.cc:
##########
@@ -60,6 +60,38 @@ void AddOne(ffi::TensorView x, ffi::TensorView y) {
// expose global symbol add_one
TVM_FFI_DLL_EXPORT_TYPED_FUNC(add_one, my_ffi_extension::AddOne);
+/*!
+ * \brief Example of a custom object that is exposed to the FFI library
+ */
+class IntPairObj : public tvm::ffi::Object {
+ public:
+ int64_t a;
+ int64_t b;
+
+ IntPairObj() = default;
+ IntPairObj(int64_t a, int64_t b) : a(a), b(b) {}
+
+ int64_t GetFirst() const { return this->a; }
+
+ // Required: declare type information
+ TVM_FFI_DECLARE_OBJECT_INFO_FINAL("my_ffi_extension.IntPair", IntPairObj,
tvm::ffi::Object);
+};
+
+/*!
+ * \brief Defines an explicit reference to IntPairObj
+ *
+ * A reference wrapper serves as a reference-counted ptr to the object.
Review Comment:
There's a typo in the comment. `ptr` should be expanded to `pointer` for
clarity.
```suggestion
* A reference wrapper serves as a reference-counted pointer to the object.
```
##########
docs/guides/packaging.md:
##########
@@ -14,21 +14,198 @@
<!--- KIND, either express or implied. See the License for the -->
<!--- specific language governing permissions and limitations -->
<!--- under the License. -->
-# Packaging
+# Python Binding and Packaging
-This guide explains how to package a tvm-ffi-based library into a Python
ABI-agnostic wheel.
-It demonstrates both source-level builds (for cross-compilation) and builds
based on pre-shipped shared libraries.
+This guide explains how to leverage tvm-ffi to expose C++ functions into
Python and package them into a wheel.
At a high level, packaging with tvm-ffi offers several benefits:
-- **ABI-agnostic wheels**: Works across different Python versions with minimal
dependency.
-- **Universally deployable**: Build once with tvm-ffi and ship to different
environments, including Python and non-Python environments.
+- **Ship one wheel** that can be used across Python versions, including
free-threaded Python.
+- **Cross language** abd accessed the binded functions any other languages,
such as Rust.
Review Comment:
This line has a couple of typos (`abd`, `binded`) and the phrasing is a bit
awkward. I suggest rephrasing for clarity and correctness. Note that markdown
formatting (bolding) has been removed from the suggestion to comply with output
requirements; please re-add it when applying the change.
```suggestion
- Cross-language access, allowing bound functions to be used from other
languages, such as Rust.
```
##########
examples/packaging/src/extension.cc:
##########
@@ -85,5 +117,15 @@ TVM_FFI_STATIC_INIT_BLOCK() {
// tvm::ffi::Module::LoadFromFile, instead, just load the dll or simply
bundle into the
// final project
refl::GlobalDef().def("my_ffi_extension.raise_error", RaiseError);
+ // register the object into the system
+ // register field accessors and a global static function `__ffi_init__` as
ffi::Function
+ refl::ObjectDef<IntPairObj>()
+ .def(refl::init<int64_t, int64_t>())
+ // Example static method that returns the first element of the pair
Review Comment:
This comment is incorrect. The `static_get_second` method returns the second
element of the pair (`pair->b`), not the first.
```suggestion
// Example static method that returns the second element of the pair
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
