gemini-code-assist[bot] commented on PR #499: URL: https://github.com/apache/tvm-ffi/pull/499#issuecomment-4028953071
## Summary of Changes Hello, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the TVM FFI system by introducing the ability to define FFI objects directly within Python using a new `@tvm_ffi.dataclasses.py_class` decorator. This change provides a more Pythonic and flexible way to create FFI-compatible data structures, moving beyond solely C++-defined objects. It involves fundamental updates to the FFI's core C++ and Cython layers to support dynamic type registration, flexible field setters (including Python functions), robust type conversion from Python type hints, and proper memory management for these new Python-defined types. The overall impact is a more seamless and powerful integration between Python and the TVM FFI, simplifying the definition and interaction with complex data structures. ### Highlights * **Python-Defined FFI Dataclasses**: Introduced `@tvm_ffi.dataclasses.py_class` decorator, enabling users to define FFI objects directly in Python with dataclass-like semantics, including automatic `__init__`, `__repr__`, `__eq__`, and `__hash__` generation. * **Flexible Field Setters**: Enhanced the FFI field system to support `FunctionObj` as setters, allowing Python functions to handle field assignment for Python-defined types. A new `kTVMFFIFieldFlagBitSetterIsFunctionObj` flag and `CallFieldSetter` utility were added to manage this. * **Unified Object Creation**: Implemented `CreateEmptyObject` and `HasCreator` functions in C++ to provide a unified mechanism for creating FFI objects, supporting both native C++ creators and Python's `__ffi_new__` type attribute. * **Robust Type Conversion**: Added a new `CAny` class in Cython to manage owned `TVMFFIAny` values and introduced comprehensive type conversion logic within `TypeSchema` (e.g., `from_annotation`, `check_value`, `convert`) to map Python type hints to FFI `TypeSchema` with eager protocol normalization. * **Deferred Registration for Forward References**: The `@py_class` decorator now supports deferred registration, allowing classes with forward references in their type annotations to be defined and resolved correctly at runtime. * **Customizable Fields**: Provided a `Field` descriptor and `field()` helper function for `@py_class` to allow fine-grained control over field properties like default values, initialization, representation, hashing, comparison, and keyword-only arguments. * **C++ Backend Support for Python Types**: Integrated C++ functions (`PyClassDeleter`, `GetFieldGetter`, `MakeFieldSetter`, `MakeFFINew`) to support the lifecycle and field access of Python-defined FFI objects, including proper memory management and type-aware field operations. 🧠**New Feature in Public Preview:** You can now enable **Memory** to help **Gemini Code Assist** learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. **Click [here](https://codeassist.google/code-review/login) to enable Memory in your admin console.** <details> <summary><b>Changelog</b></summary> * **include/tvm/ffi/c_api.h** * Added `kTVMFFIFieldFlagBitSetterIsFunctionObj` flag to `TVMFFIFieldFlagBitMask`. * Changed `TVMFFIFieldInfo::setter` type from `TVMFFIFieldSetter` to `void*` to accommodate `FunctionObj`. * **include/tvm/ffi/function.h** * Added `CreateEmptyObject` function to unify object creation from native creators or `__ffi_new__`. * Added `HasCreator` function to check if a type supports reflection creation. * **include/tvm/ffi/reflection/accessor.h** * Introduced `CallFieldSetter` to dispatch field setting based on `kTVMFFIFieldFlagBitSetterIsFunctionObj`. * Updated `FieldSetter::operator()` and `SetFieldToDefault` to use `CallFieldSetter`. * **include/tvm/ffi/reflection/creator.h** * Modified `ObjectCreator` constructor and `operator()` to utilize `HasCreator` and `CreateEmptyObject`. * **include/tvm/ffi/reflection/init.h** * Included `tvm/ffi/cast.h` header. * Added `details::CastFromAny` template for type-safe `AnyView` to `ObjectRef` conversion. * Updated `MakeInit` to check for creators using `HasCreator` and create objects using `CreateEmptyObject`. * Corrected namespace for `TypeSchemaImpl` in `RegisterAutoInit`. * **include/tvm/ffi/reflection/overload.h** * Added global namespace qualifiers (`::tvm::ffi::details`) to various type aliases and function calls to resolve potential ambiguity. * **include/tvm/ffi/reflection/registry.h** * Added global namespace qualifiers (`::tvm::ffi::details`) to various type aliases and function calls. * Introduced `kConvert` type attribute for `AnyView` to reflected object conversion. * Added `RegisterConvertTypeAttr` template function to register `__ffi_convert__` for object reference types. * Added `ObjectDef::ref()` method to register `__ffi_convert__` for object reference wrappers. * **python/tvm_ffi/__init__.py** * Imported `CAny` from `core`. * Imported `KW_ONLY`, `Field`, `field`, and `py_class` from `dataclasses`. * **python/tvm_ffi/core.pyi** * Added `_register_py_class` and `_register_fields` function stubs. * Added `CAny` class definition with `__init__`, `type_index`, and `to_py` methods. * Extended `TypeSchema` with `origin_type_index`, `from_type_index`, `from_annotation`, `check_value`, `convert`, and `to_json` methods. * Added `ty` attribute to `TypeField`. * Added `_register_fields` method to `TypeInfo`. * **python/tvm_ffi/cython/base.pxi** * Added `cause_chain` and `extra_context` to `TVMFFIObject` struct. * Added new field flags (`kTVMFFIFieldFlagBitMaskSEqHashIgnore`, `kTVMFFIFieldFlagBitMaskSEqHashDef`, `kTVMFFIFieldFlagBitMaskReprOff`, `kTVMFFIFieldFlagBitMaskCompareOff`, `kTVMFFIFieldFlagBitMaskHashOff`, `kTVMFFIFieldFlagBitSetterIsFunctionObj`). * Added `TVMFFISEqHashKind` enum. * Changed `TVMFFITypeMetadata::total_size` to `int32_t` and added `structural_eq_hash_kind`. * Changed `TVMFFIFieldInfo::setter` to `void*`. * Added new C API functions for type and field registration (`TVMFFITypeGetOrAllocIndex`, `TVMFFITypeRegisterField`, `TVMFFITypeRegisterMetadata`, `TVMFFITypeRegisterAttr`, `TVMFFIErrorSetRaisedFromCStr`). * Modified `TVMFFIPyCallFieldSetter` signature and implementation to include `field_flags` and handle `FunctionObj` setters. * **python/tvm_ffi/cython/core.pyx** * Included `type_converter.pxi`. * **python/tvm_ffi/cython/object.pxi** * Added `flags` attribute to `FieldSetter`. * Implemented `_register_py_class` for registering Python-defined FFI types. * Implemented `_rollback_py_class` to undo Python-level type registration. * Modified `_lookup_type_attr` to use `TVMFFIAnyViewToOwnedAny`. * Implemented `CAny` class for managing owned `TVMFFIAny` values. * **python/tvm_ffi/cython/tvm_ffi_python_helpers.h** * Modified `TVMFFIPyCallManager::SetField` to accept `field_flags` and dispatch between function pointer and `FunctionObj` setters. * Updated `TVMFFIPyCallFieldSetter` signature to include `field_flags`. * **python/tvm_ffi/cython/type_converter.pxi** * Added new file `type_converter.pxi`. * Implemented `_TypeConverter` class and various `_tc_convert_*` functions for type-aware conversion of Python values to `CAny` based on `TypeSchema`. * **python/tvm_ffi/cython/type_info.pxi** * Imported `typing`, `collections.abc`, and `cached_property`. * Added `_UnionType` handling for Python 3.10+. * Added `flags` to `FieldSetter`. * Extended `_TYPE_SCHEMA_ORIGIN_CONVERTER` with C++ STL types and `ObjectRValueRef` mappings. * Introduced `_ORIGIN_TO_TYPE_INDEX` and `_TYPE_INDEX_TO_ORIGIN` maps for direct type index lookup. * Extended `TypeSchema` with `origin_type_index`, `_converter` (cached property), `from_type_index`, `from_annotation`, `check_value`, `convert`, and `to_json` methods. * Modified `TypeSchema.__post_init__` to handle `args` and resolve `origin_type_index`. * Modified `TypeSchema.repr` to handle `schema_args` correctly. * Added `ty` attribute to `TypeField`. * Added `total_size` cached property to `TypeInfo`. * Implemented `_register_fields` and `_register_methods` for `TypeInfo` to handle Python-defined type fields. * Added `_ORIGIN_NATIVE_LAYOUT` for field size/alignment information. * Implemented `_register_one_field` and `_f_type_convert` (C callback) for field registration details. * **python/tvm_ffi/dataclasses/__init__.py** * Imported `KW_ONLY`, `Field`, `field`, and `py_class`. * Updated `__all__` to include new dataclass components. * **python/tvm_ffi/dataclasses/field.py** * Added new file `field.py`. * Defined `KW_ONLY` sentinel for Python < 3.10. * Implemented `Field` class as a descriptor for Python-defined FFI type fields. * Implemented `field()` helper function for customizing `Field` instances. * **python/tvm_ffi/dataclasses/py_class.py** * Added new file `py_class.py`. * Implemented `@py_class` decorator for Python-defined FFI classes with dataclass semantics. * Introduced two-phase registration (`_phase1_register_type`, `_phase2_register_fields`) to handle forward references. * Added `_PendingClass` and `_PY_CLASS_BY_MODULE` for deferred resolution. * Implemented `_collect_own_fields` to parse annotations and `Field` metadata. * Implemented `_flush_pending`, `_raise_unresolved_forward_reference`, `_make_temporary_init`, and `_install_deferred_init` for deferred registration logic. * **python/tvm_ffi/registry.py** * Modified `_make_init` to conditionally call `__post_init__` if present in the class. * Updated `_install_dataclass_dunders` to respect user-defined overrides for `__eq__`, `__ne__`, and ordering methods. * **rust/tvm-ffi-sys/src/c_api.rs** * Updated `TVMFFIFieldInfo::setter` to be a `*mut c_void` and added documentation for its dual purpose (function pointer or `FunctionObj` handle). * **src/ffi/extra/dataclass.cc** * Added `PyClassDeleter` for Python-defined object memory management. * Implemented `PyClassFieldGetter` template for generic field retrieval. * Implemented `GetFieldGetter` to return appropriate getter function pointers. * Implemented `WriteFieldValue` to write converted values to fields. * Implemented `MakeFieldSetter` to create `FunctionObj` setters for Python-defined fields, integrating with Cython's type conversion. * Implemented `MakeFFINew` to register `__ffi_new__` and `__ffi_shallow_copy__` for Python-defined types. * Registered new FFI functions: `ffi.GetFieldGetter`, `ffi.MakeFieldSetter`, `ffi.MakeFFINew`, `ffi.RegisterAutoInit`. * **src/ffi/extra/reflection_extra.cc** * Updated `MakeObjectFromPackedArgs` to use `CreateEmptyObject` and `reflection::CallFieldSetter`. * **src/ffi/extra/serialization.cc** * Updated `ObjectGraphDeserializer` to use `CreateEmptyObject` and `reflection::CallFieldSetter`. * **src/ffi/function.cc** * Added `ffi.FunctionFromExternC` to create `Function` from external C function pointers. * Updated `ffi.GetGlobalFuncMetadata` to be a lambda. * **src/ffi/object.cc** * Included new container headers: `dict.h`, `list.h`, `tensor.h`. * Added destructor `~Entry()` to `TypeTable::Entry` to release `FunctionObj` setters. * Modified `TypeTable::RegisterTypeField` to `IncRef` `FunctionObj` setters. * Added `structural_eq_hash_kind` to `TVMFFITypeMetadata` for `Object` type. * Registered `__ffi_convert__` type attribute for various core FFI object types (e.g., `ObjectRef`, `String`, `Array`, `Map`). * **src/ffi/testing/testing.cc** * Added `.ref<TestIntPair>()` to `ObjectDef<TestIntPairObj>` to register `__ffi_convert__`. </details> <details> <summary><b>Using Gemini Code Assist</b></summary> <br> The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips. <b>Invoking Gemini</b> You can request assistance from Gemini at any point by creating a comment using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands on the current page. Feature | Command | Description --- | --- | --- Code Review | `/gemini review` | Performs a code review for the current pull request in its current state. Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state. Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments. Help | `/gemini help` | Displays a list of available commands. <b>Customization</b> To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). <b>Limitations & Feedback</b> Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG). <b>You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist).</b> </details> [^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
