tqchen commented on code in PR #191: URL: https://github.com/apache/tvm-ffi/pull/191#discussion_r2463907538
########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result Review Comment: TVMFFIAny* ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ Review Comment: Managed Tensor objects that follows DLPack layout. "DLPack tensors can be confusing since ppl may mistake it with DLManagedTensor" ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. Review Comment: a library handle that contains the function to be called. It is usually `nullptr` for exported symbols. It can be used to enable closures that captures extra data. ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] Review Comment: const TVMFFIAny* ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. Review Comment: reserved used for small string length and padded to zero in other cases ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` Review Comment: TVMFFIAny ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: Review Comment: TVM FFI defines a common stable C ABI protocol for all functions ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. +- ``args: Any*``: a pointer to an array of ``Any`` values for input arguments, stored contiguously on stack or heap. +- ``num_args: int``: number of input arguments. +- ``result: Any*``: a pointer to a single ``Any`` that receives the output result. + +.. figure:: ../_static/tvm-ffi-layout-func.svg + :alt: Calling convention for universal_c_abi + :name: fig:layout-func + + Figure 2. Layout spec of ``universal_c_abi`` function. + + +Stability and Interoperability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Review Comment: likely this section is redundant with readme and can be removed ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea Review Comment: TVM FFI Function C ABI ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. Review Comment: TVMFFIAny ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure Review Comment: tvm_ffi_func_c_abi ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. +- ``args: Any*``: a pointer to an array of ``Any`` values for input arguments, stored contiguously on stack or heap. +- ``num_args: int``: number of input arguments. +- ``result: Any*``: a pointer to a single ``Any`` that receives the output result. + +.. figure:: ../_static/tvm-ffi-layout-func.svg + :alt: Calling convention for universal_c_abi + :name: fig:layout-func + + Figure 2. Layout spec of ``universal_c_abi`` function. + + +Stability and Interoperability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Stability.** The pure C layout and the calling convention are stable across compiler versions and independent of host languages or frameworks. + +**Cross-language.** TVM-FFI implements this calling convention in multiple languages (C, C++, Python, Rust, ...), enabling code written in one language—or generated by a DSL targeting the ABI—to be called from another language. + +**Cross-framework.** TVM-FFI uses standard data structures such as `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ to represent arrays, so compiled functions can be used from any array framework that implements the DLPack protocol (NumPy, PyTorch, TensorFlow, CuPy, JAX, and others). + + +Stable ABI in C Code +-------------------- + Review Comment: We design the C ABI with DSL and ML compilers in mind. In the context of ML compilers, usually we do not have access to c++ templates and needs to operate directly in LLVM (MLIR dialect) or low-level C. In such cases, it is helpful to directly leverage and target the stable C ABI. This section shows how we can expose a kernel directly using the C ABI. Because we write code directly in C without any boilerplate, the code can serve as a direct reference for DSL compilers to expose kernel functions following our C ABI standard. ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. +- ``args: Any*``: a pointer to an array of ``Any`` values for input arguments, stored contiguously on stack or heap. +- ``num_args: int``: number of input arguments. +- ``result: Any*``: a pointer to a single ``Any`` that receives the output result. + +.. figure:: ../_static/tvm-ffi-layout-func.svg + :alt: Calling convention for universal_c_abi + :name: fig:layout-func + + Figure 2. Layout spec of ``universal_c_abi`` function. + + +Stability and Interoperability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Stability.** The pure C layout and the calling convention are stable across compiler versions and independent of host languages or frameworks. + +**Cross-language.** TVM-FFI implements this calling convention in multiple languages (C, C++, Python, Rust, ...), enabling code written in one language—or generated by a DSL targeting the ABI—to be called from another language. + +**Cross-framework.** TVM-FFI uses standard data structures such as `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ to represent arrays, so compiled functions can be used from any array framework that implements the DLPack protocol (NumPy, PyTorch, TensorFlow, CuPy, JAX, and others). + + +Stable ABI in C Code +-------------------- + +.. note:: + + All the code in this tutorial can be found under `examples/stable_c_abi <https://github.com/apache/tvm-ffi/tree/main/examples/stable_c_abi>`_ in the repository, including commands to compile the code and run the examples. + +This section shows how to write C code that follows the stable C ABI. Specifically, we provide two examples: + +- A CPU ``add_one_cpu`` kernel in C that is equivalent to the :ref:`C++ example <cpp_add_one_kernel>`. Review Comment: change name to `add_one_c` or `add_one_c_abi` ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. +- ``args: Any*``: a pointer to an array of ``Any`` values for input arguments, stored contiguously on stack or heap. +- ``num_args: int``: number of input arguments. +- ``result: Any*``: a pointer to a single ``Any`` that receives the output result. + +.. figure:: ../_static/tvm-ffi-layout-func.svg + :alt: Calling convention for universal_c_abi + :name: fig:layout-func + + Figure 2. Layout spec of ``universal_c_abi`` function. + + +Stability and Interoperability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Stability.** The pure C layout and the calling convention are stable across compiler versions and independent of host languages or frameworks. + +**Cross-language.** TVM-FFI implements this calling convention in multiple languages (C, C++, Python, Rust, ...), enabling code written in one language—or generated by a DSL targeting the ABI—to be called from another language. + +**Cross-framework.** TVM-FFI uses standard data structures such as `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ to represent arrays, so compiled functions can be used from any array framework that implements the DLPack protocol (NumPy, PyTorch, TensorFlow, CuPy, JAX, and others). + + +Stable ABI in C Code +-------------------- + +.. note:: + + All the code in this tutorial can be found under `examples/stable_c_abi <https://github.com/apache/tvm-ffi/tree/main/examples/stable_c_abi>`_ in the repository, including commands to compile the code and run the examples. + +This section shows how to write C code that follows the stable C ABI. Specifically, we provide two examples: + +- A CPU ``add_one_cpu`` kernel in C that is equivalent to the :ref:`C++ example <cpp_add_one_kernel>`. +- A loader and runner in C that invokes the kernel, a direct C translation of the :ref:`C++ example <cpp_load>`. + +The first shows **how to write a function** that follows the stable C ABI; the second shows **how to call a function** via the stable C ABI. + +C Kernel ``add_one_cpu`` +~~~~~~~~~~~~~~~~~~~~~~~~ + +Below is a minimal ``add_one_cpu`` kernel in C that follows the stable C ABI. It has three steps: + +- **Step 1**. Extract input ``x`` and output ``y`` as DLPack tensors; +- **Step 2**. Implement the kernel ``y = x + 1`` on CPU with a simple for-loop; +- **Step 3**. Set the output result to ``result``. + +.. literalinclude:: ../../examples/stable_c_abi/src/add_one_cpu.c + :language: c + :start-after: [example.begin] + :end-before: [example.end] + + +Compared to the :ref:`C++ example <cpp_add_one_kernel>`, there are a few key differences: + +- The explicit marshalling in **Step 1** is only needed in C. In C++, templates hide these details. +- The C++ macro :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` (used to export ``add_one_cpu``) is not needed in C, because this example directly defines the exported C symbol ``__tvm_ffi_add_one_cpu``. + + +.. hint:: + + In TVM-FFI's C++ APIs, many invocables (functions, lambdas, functors) are automatically converted into the universal C ABI form by :cpp:class:`tvm::ffi::Function` and :cpp:class:`tvm::ffi::TypedFunction`. + + Rule of thumb: if an invocable's arguments and result can be converted to/from :cpp:class:`tvm::ffi::Any`, it can be wrapped as a universal C ABI function. + + +Load and Run the Kernel +~~~~~~~~~~~~~~~~~~~~~~~ + +Next, a minimal C loader invokes the ``add_one_cpu`` kernel. It is functionally identical to the :ref:`C++ example <cpp_load>` and performs: Review Comment: C loader is less useeful, instead we can use a python loader. Since the main goal is to demonstrate we can use the same C ABI to expose functions that can be universally accessed. ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings Review Comment: Other objects( C-style string can cause some confusion), since string can either be small string(on-stack) or on heap, it is ok to remove this ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] Review Comment: It is better to be consistent with the C ABI code, the simplification is less useful and could cause confusion with c++ ffi::Any(note that they re managed and unmanged version) ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. +- ``args: Any*``: a pointer to an array of ``Any`` values for input arguments, stored contiguously on stack or heap. +- ``num_args: int``: number of input arguments. +- ``result: Any*``: a pointer to a single ``Any`` that receives the output result. + +.. figure:: ../_static/tvm-ffi-layout-func.svg + :alt: Calling convention for universal_c_abi + :name: fig:layout-func + + Figure 2. Layout spec of ``universal_c_abi`` function. + + +Stability and Interoperability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Stability.** The pure C layout and the calling convention are stable across compiler versions and independent of host languages or frameworks. + +**Cross-language.** TVM-FFI implements this calling convention in multiple languages (C, C++, Python, Rust, ...), enabling code written in one language—or generated by a DSL targeting the ABI—to be called from another language. + +**Cross-framework.** TVM-FFI uses standard data structures such as `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ to represent arrays, so compiled functions can be used from any array framework that implements the DLPack protocol (NumPy, PyTorch, TensorFlow, CuPy, JAX, and others). + + +Stable ABI in C Code +-------------------- + +.. note:: + + All the code in this tutorial can be found under `examples/stable_c_abi <https://github.com/apache/tvm-ffi/tree/main/examples/stable_c_abi>`_ in the repository, including commands to compile the code and run the examples. + +This section shows how to write C code that follows the stable C ABI. Specifically, we provide two examples: + +- A CPU ``add_one_cpu`` kernel in C that is equivalent to the :ref:`C++ example <cpp_add_one_kernel>`. +- A loader and runner in C that invokes the kernel, a direct C translation of the :ref:`C++ example <cpp_load>`. + +The first shows **how to write a function** that follows the stable C ABI; the second shows **how to call a function** via the stable C ABI. + +C Kernel ``add_one_cpu`` +~~~~~~~~~~~~~~~~~~~~~~~~ + +Below is a minimal ``add_one_cpu`` kernel in C that follows the stable C ABI. It has three steps: + +- **Step 1**. Extract input ``x`` and output ``y`` as DLPack tensors; +- **Step 2**. Implement the kernel ``y = x + 1`` on CPU with a simple for-loop; +- **Step 3**. Set the output result to ``result``. + +.. literalinclude:: ../../examples/stable_c_abi/src/add_one_cpu.c + :language: c + :start-after: [example.begin] + :end-before: [example.end] + + +Compared to the :ref:`C++ example <cpp_add_one_kernel>`, there are a few key differences: + +- The explicit marshalling in **Step 1** is only needed in C. In C++, templates hide these details. +- The C++ macro :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` (used to export ``add_one_cpu``) is not needed in C, because this example directly defines the exported C symbol ``__tvm_ffi_add_one_cpu``. + + +.. hint:: + + In TVM-FFI's C++ APIs, many invocables (functions, lambdas, functors) are automatically converted into the universal C ABI form by :cpp:class:`tvm::ffi::Function` and :cpp:class:`tvm::ffi::TypedFunction`. + + Rule of thumb: if an invocable's arguments and result can be converted to/from :cpp:class:`tvm::ffi::Any`, it can be wrapped as a universal C ABI function. + + +Load and Run the Kernel +~~~~~~~~~~~~~~~~~~~~~~~ + +Next, a minimal C loader invokes the ``add_one_cpu`` kernel. It is functionally identical to the :ref:`C++ example <cpp_load>` and performs: + +- **Step 1**. Load the shared library ``build/add_one_cpu.so`` that contains the kernel; +- **Step 2**. Get function ``add_one_cpu`` from the library; +- **Step 3**. Invoke the function with two `DLTensor <https://dmlc.github.io/dlpack/latest/c_api.html#c.DLTensor>`_ inputs ``x`` and ``y``; + +.. literalinclude:: ../../examples/stable_c_abi/src/load.c + :language: c + :start-after: [main.begin] + :end-before: [main.end] + + +.. dropdown:: Auxiliary Logics + + .. literalinclude:: ../../examples/stable_c_abi/src/load.c + :language: c + :start-after: [aux.begin] + :end-before: [aux.end] + +To call a function via the stable C ABI in C, idiomatically: + +- Convert input arguments to the ``Any`` type; +- Call the target function (e.g., ``add_one_cpu``) via :cpp:func:`TVMFFIFunctionCall`; +- Optionally convert the output ``Any`` back to the desired type, if the function returns a value. + +What's Next +----------- + +**ABI specification.** See the complete ABI specification in :doc:`../concepts/abi_overview`. + +**Convenient compiler target.** The stable C ABI is a simple, portable codegen target for DSL compilers. Emit C that follows this ABI to integrate with TVM-FFI and call the result from multiple languages and frameworks. See :doc:`../guides/compiler_integration`. + +**Rich and extensible type system.** TVM-FFI supports a rich set of types in the stable C ABI: primitive types (integers, floats), DLPack tensors, strings, built-in reference-counted objects (functions, arrays, maps), and user-defined reference-counted objects. See :doc:`../guides/cpp_guide`. Review Comment: Likely the two we should recommend reads are abi specification and compiler_integration ########## docs/get_started/stable_c_abi.rst: ########## @@ -0,0 +1,189 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at +.. +.. http://www.apache.org/licenses/LICENSE-2.0 +.. +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +Stable C ABI +============ + +TVM-FFI centers on a single key idea: + + +.. admonition:: Key Idea + :class: important + + Every function call can be represented by a single ABI stable C function: + + .. code-block:: c + + int universal_c_abi( // returns 0 if succeed, error code if failure + void* handle, // library handle + Any* args, // inputs: args[0 ... N - 1] + int N, // number of inputs + Any* result, // output: *result + ); + + where ``Any`` is a tagged union of all supported types, e.g. integers, floats, Tensors, strings, etc., and can be further extended to arbitrary user-defined types. + +Built on top of this stable C ABI, TVM-FFI provides an extensible, performant, and ecosystem-friendly open solution for all. + +The rest of this guide covers: + +- The stable C layout and calling convention of ``universal_c_abi``; +- C examples that implements the stable C ABI, and calls functions that follow this ABI. + +Stable C Layout +--------------- + +The ``universal_c_abi`` function uses a stable layout for all the input and output arguments. + +Layout of ``Any`` +~~~~~~~~~~~~~~~~~ + +TVM-FFI's ``Any`` is a fixed size (128-bit) tagged union that represents all supported types. + +- First 32 bits: type index indicating which value is stored (supports up to 2^32 types). +- Next 32 bits: null padding, or reserved for special flags in rare cases. +- Last 64 bits: payload that is either a 64-bit integer, a 64-bit floating-point number, or a pointer to a heap-allocated object. + +.. figure:: ../_static/tvm-ffi-layout-any.svg + :alt: Layout of the 128-bit Any tagged union + :name: fig:layout-any + + Figure 1. Layout spec for the ``Any`` type. + +The following conventions apply when representing values in ``Any``: + +- Primitive types: the last 64 bits directly store the value, for example: + + * Integers + * Floating-point numbers + +- Heap-allocated objects: the last 64 bits store a pointer to the actual object, for example: + + * `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ + * C-style strings + +- Arbitrary objects: the type index identifies the concrete type, and the last 64 bits store a pointer to a reference-counted object in TVM-FFI's object format, for example: + + * :py:class:`tvm_ffi.Function`, representing all functions, such as Python/C++ functions/lambdas, etc.; + * :py:class:`tvm_ffi.Array` and :py:class:`tvm_ffi.Map` as list/dictionary containers of all ``Any`` values; + * Up to 2^32 types can be represented in this type system. + +Function Calling Convention +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Function calls in TVM-FFI share the same calling convention, ``universal_c_abi``, as described above. + +- ``handle: void*``: a library handle that contains the function to be called. It is usually ``nullptr`` for most use cases, but some libraries (such as cuDNN) may use it to store global contexts. +- ``args: Any*``: a pointer to an array of ``Any`` values for input arguments, stored contiguously on stack or heap. +- ``num_args: int``: number of input arguments. +- ``result: Any*``: a pointer to a single ``Any`` that receives the output result. + +.. figure:: ../_static/tvm-ffi-layout-func.svg + :alt: Calling convention for universal_c_abi + :name: fig:layout-func + + Figure 2. Layout spec of ``universal_c_abi`` function. + + +Stability and Interoperability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**Stability.** The pure C layout and the calling convention are stable across compiler versions and independent of host languages or frameworks. + +**Cross-language.** TVM-FFI implements this calling convention in multiple languages (C, C++, Python, Rust, ...), enabling code written in one language—or generated by a DSL targeting the ABI—to be called from another language. + +**Cross-framework.** TVM-FFI uses standard data structures such as `DLPack tensors <https://data-apis.org/array-api/2024.12/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ to represent arrays, so compiled functions can be used from any array framework that implements the DLPack protocol (NumPy, PyTorch, TensorFlow, CuPy, JAX, and others). + + +Stable ABI in C Code +-------------------- + +.. note:: + + All the code in this tutorial can be found under `examples/stable_c_abi <https://github.com/apache/tvm-ffi/tree/main/examples/stable_c_abi>`_ in the repository, including commands to compile the code and run the examples. + +This section shows how to write C code that follows the stable C ABI. Specifically, we provide two examples: + +- A CPU ``add_one_cpu`` kernel in C that is equivalent to the :ref:`C++ example <cpp_add_one_kernel>`. +- A loader and runner in C that invokes the kernel, a direct C translation of the :ref:`C++ example <cpp_load>`. + +The first shows **how to write a function** that follows the stable C ABI; the second shows **how to call a function** via the stable C ABI. + +C Kernel ``add_one_cpu`` +~~~~~~~~~~~~~~~~~~~~~~~~ + +Below is a minimal ``add_one_cpu`` kernel in C that follows the stable C ABI. It has three steps: + +- **Step 1**. Extract input ``x`` and output ``y`` as DLPack tensors; +- **Step 2**. Implement the kernel ``y = x + 1`` on CPU with a simple for-loop; +- **Step 3**. Set the output result to ``result``. + +.. literalinclude:: ../../examples/stable_c_abi/src/add_one_cpu.c + :language: c + :start-after: [example.begin] + :end-before: [example.end] + + +Compared to the :ref:`C++ example <cpp_add_one_kernel>`, there are a few key differences: + +- The explicit marshalling in **Step 1** is only needed in C. In C++, templates hide these details. Review Comment: it is better to directly talk about this example, target audience here is compiler developers, we can add c++ template reference in end. At high-level, we perform the following steps: - Function symbols follow name int __tvm_ffi_<name> - We go over and check the type_index in TVMFFIAny and extract the input function arguments - The function Return 0 for success, -1 for error - When error happens, we set the error via `TVMFFIErrorSetRaisedFromCStr` or `TVMFFIErrorSetRaisedFromCStrParts` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
