gemini-code-assist[bot] commented on PR #36:
URL: https://github.com/apache/tvm-ffi/pull/36#issuecomment-3367433723
This PR introduces a comprehensive system for generating and attaching
JSON-based type schemas to FFI-exposed entities in TVM, including global
functions, object fields, and object methods. This enhancement provides richer,
machine-readable type information, which can be invaluable for tooling, static
analysis, and dynamic dispatch mechanisms built on top of the TVM FFI.
**Key Changes:**
1. **JSON-based Type Schema Generation (C++):**
* Introduced `tvm::ffi::details::TypeSchema<T>` template to generate a
JSON string representing the type `T`. This is implemented for various
fundamental types (e.g., `int`, `float`, `bool`, `String`, `Bytes`,
`DLDataType`, `DLDevice`) and complex container types (e.g., `Array<T>`,
`Map<K, V>`, `Optional<T>`, `Tuple<Args...>`, `Variant<Args...>`).
* For functions, `tvm::ffi::TypedFunction<R(Args...)>::TypeSchema()`
and `tvm::ffi::details::FuncFunctorImpl` now provide JSON schemas that include
return type and argument types.
* A new `tvm::ffi::EscapeString` utility is added for proper JSON
string escaping.
2. **Enhanced Reflection Metadata (C++):**
* The `tvm::ffi::reflection::Metadata` class is introduced, allowing
arbitrary key-value pairs to be attached to fields and methods during
registration.
* `FieldInfoBuilder` and `MethodInfoBuilder` are added to temporarily
hold this metadata before it's serialized into the `TVMFFIFieldInfo` and
`TVMFFIMethodInfo` C structs.
* The `type_schema` generated by `TypeSchema<T>` is automatically
included as a metadata entry for all registered fields, methods, and global
functions.
3. **Python API for Schema Access:**
* A new `tvm_ffi.core.TypeSchema` Python class is added, capable of
parsing the JSON schema strings into a structured Python object, providing a
more convenient way to inspect type information.
* `tvm_ffi.core.TypeField` and `tvm_ffi.core.TypeMethod` now include a
`metadata` dictionary, making the attached JSON metadata accessible from Python.
* A new Python function `tvm_ffi.get_global_func_metadata(name: str)`
is exposed to retrieve the full metadata (including `type_schema`) for any
registered global function.
* Registered Python classes now have a `__tvm_ffi_type_info__`
attribute, providing access to their fields and methods, each with their
associated metadata.
**Concrete Examples:**
**C++ Examples:**
```cpp
#include <tvm/ffi/container/array.h>
#include <tvm/ffi/container/map.h>
#include <tvm/ffi/reflection/registry.h>
#include <tvm/ffi/string.h>
#include <tvm/ffi/type_traits.h>
namespace tvm {
namespace ffi {
namespace reflection {
// Example 1: Generating a type schema for a simple type
std::string int_schema = details::TypeSchema<int>::v();
// int_schema will be: "{\"type\":\"int\"}"
// Example 2: Generating a type schema for a complex type
std::string array_int_schema = details::TypeSchema<Array<int64_t>>::v();
// array_int_schema will be:
"{\"type\":\"ffi.Array\",\"args\":[{\"type\":\"int\"}]}"
// Example 3: Registering a global function with custom metadata and type
schema
int64_t my_add_func(int64_t a, int64_t b) { return a + b; }
TVM_FFI_STATIC_INIT_BLOCK() {
GlobalDef()
.def("my_add", my_add_func,
Metadata{{"description", "Adds two integers"}, {"version", 1}});
}
// The 'type_schema' for "my_add" will be automatically generated and
included in its metadata.
// You can retrieve it using the ffi.GetGlobalFuncMetadata C++ API (exposed
to Python).
// Example 4: Registering a field with custom metadata and type schema
class MyObjectObj : public Object {
public:
int my_field;
TVM_FFI_DECLARE_OBJECT_INFO("my.MyObject", MyObjectObj, Object);
};
class MyObject : public ObjectRef {
public:
TVM_FFI_DEFINE_OBJECT_REF_METHODS_NULLABLE(MyObject, ObjectRef,
MyObjectObj);
};
TVM_FFI_STATIC_INIT_BLOCK() {
ObjectDef<MyObjectObj>()
.def_rw("my_field", &MyObjectObj::my_field, "A simple integer field",
Metadata{{"unit", "count"}});
}
// The 'type_schema' for "my_field" will be automatically generated and
included in its metadata.
// You can retrieve it using GetFieldInfo (as shown in
tests/cpp/test_metadata.cc).
} // namespace reflection
} // namespace ffi
} // namespace tvm
```
**Python Examples:**
```python
import json
from typing import Any, Callable
from tvm_ffi import get_global_func_metadata, register_object
from tvm_ffi.core import TypeInfo, TypeSchema
# Example 1: Accessing global function metadata and type schema
# (Assuming 'testing.schema_id_int' is registered in C++ as in
src/ffi/extra/testing.cc)
metadata: dict[str, Any] = get_global_func_metadata("testing.schema_id_int")
# metadata will contain (example values):
# {
# "type_schema":
"{\"type\":\"ffi.Function\",\"args\":[{\"type\":\"int\"},{\"type\":\"int\"}]}",
# "bool_attr": True,
# "int_attr": 1,
# "str_attr": "hello"
# }
schema_str: str = metadata["type_schema"]
type_schema_obj: TypeSchema = TypeSchema.from_json_str(schema_str)
print(f"Schema for testing.schema_id_int: {type_schema_obj}")
# Expected output: Schema for testing.schema_id_int: Callable[[int], int]
# Example 2: Accessing object field metadata and type schema
@register_object("testing.MyPythonObject")
class MyPythonObject:
my_int_field: int
my_str_field: str
# After registration, type_info is available
type_info: TypeInfo = getattr(MyPythonObject, "__tvm_ffi_type_info__")
# Find the field "my_int_field"
int_field_schema_str = ""
for field in type_info.fields:
if field.name == "my_int_field":
int_field_schema_str = field.metadata["type_schema"]
break
int_field_type_schema = TypeSchema.from_json_str(int_field_schema_str)
print(f"Schema for MyPythonObject.my_int_field: {int_field_type_schema}")
# Expected output: Schema for MyPythonObject.my_int_field: int
# Example 3: Accessing object method metadata and type schema
# (Assuming 'add_int' is a method registered on 'testing.SchemaAllTypes' in
C++ as in src/ffi/extra/testing.cc)
from tvm_ffi.testing import _SchemaAllTypes
schema_all_types_info: TypeInfo = getattr(_SchemaAllTypes,
"__tvm_ffi_type_info__")
add_int_method_schema_str = ""
for method in schema_all_types_info.methods:
if method.name == "add_int":
add_int_method_schema_str = method.metadata["type_schema"]
break
add_int_method_type_schema =
TypeSchema.from_json_str(add_int_method_schema_str)
print(f"Schema for _SchemaAllTypes.add_int: {add_int_method_type_schema}")
# Expected output: Schema for _SchemaAllTypes.add_int: Callable[[int], int]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
