This is an automated email from the ASF dual-hosted git repository.
tlopex pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tvm-ffi.git
The following commit(s) were added to refs/heads/main by this push:
new 24cfa44c [STUBGEN][RUST] Generate complete object mirrors and
allocators (#740)
24cfa44c is described below
commit 24cfa44ca425ece2f2547ae87b81fea2835ddebd
Author: Linzhang Li <[email protected]>
AuthorDate: Fri Sep 4 17:58:14 2026 -0400
[STUBGEN][RUST] Generate complete object mirrors and allocators (#740)
## Summary
Attach the layout classifier (#730) to the Rust backend. A type whose
layout the registry proves is now bound *complete*: a `#[repr(C)]`
struct with every physical field at its reflected offset and width, a
`const` size/alignment assertion, and a lossless allocator
(`<Leaf>Obj::new` crate-private, `<Leaf>::new` public). Everything else
keeps the opaque form of #738. Three new directives: `opaque` vetoes a
reproducible layout, `upcast` adds a hand-written typed view,
`custom-new` renames the generated allocator to `from_complete_fields`
so a hand-written `new` can own the name.
## Motivation
This is what tvm-rust-ext's `STUBGEN_FEEDBACK.md` asks for: ordinary
data nodes allocated in Rust from their complete fields, polymorphic and
unreflected-byte types kept opaque. Generating `ir.` / `tirx.` /
`arith.` / `target.` from tvm-rust-ext's `libtvm_compiler.so` gives 131
types (98 complete, 33 opaque) that compile against the crate without
edits; the differences from the hand-written bindings are exactly what
the directives cover.
## Changes
- `stub/layout.py`: `classify(..., unmirrored=...)` and the `no-mirror`
reason, so types under a builtin parent (`ffi.Enum`, ...) stay opaque:
their base is only a header-only stand-in.
- `rust_generator/codegen.py`: classify each object with its ancestors;
mirror fields (scalars by reflected width, `Optional<T>` as `Option<T>`
or `tvm_ffi::Optional<T>` by payload, directive widths checked against
the field size); render the complete struct and its allocators; `upcast`
and `custom-new` handling.
- `rust_generator/directives.py`, `consts.py`: the three directives and
the width tables.
- `rust_generator/utils.py`: a reflected field named `base` or `data` is
spelled `base_` / `data_`, since those are the generated struct's own
members (TVM has `tirx.Ramp.base`, `tirx.DeclBuffer.data`).
- `examples/rust_stubgen/`: `IntPair` is now a plain data object,
allocated from Rust with the generated `IntPair::new` and read back by
C++.
## Testing
`test_stubgen_rust.py` (36 cases), `test_stubgen.py`,
`test_stub_layout.py`: 121 passed; ruff clean. The example regenerates
an identical `mod.rs` and `cargo run` prints `a=1 b=2 kind=PairKind(1)`
and `sum=3`.
---------
Signed-off-by: yuchuan <[email protected]>
---
examples/rust_stubgen/README.md | 28 +-
.../rust/src/generated/rust_stubgen/mod.rs | 33 +-
examples/rust_stubgen/rust/src/main.rs | 15 +-
examples/rust_stubgen/src/int_pair.cc | 16 +-
python/tvm_ffi/stub/layout.py | 16 +-
python/tvm_ffi/stub/rust_generator/codegen.py | 328 ++++++++--
python/tvm_ffi/stub/rust_generator/consts.py | 50 +-
python/tvm_ffi/stub/rust_generator/directives.py | 49 +-
python/tvm_ffi/stub/rust_generator/utils.py | 8 +-
tests/python/test_stub_layout.py | 10 +
tests/python/test_stubgen_rust.py | 664 ++++++++++++++++++---
11 files changed, 1019 insertions(+), 198 deletions(-)
diff --git a/examples/rust_stubgen/README.md b/examples/rust_stubgen/README.md
index 080b5d6a..42b2758c 100644
--- a/examples/rust_stubgen/README.md
+++ b/examples/rust_stubgen/README.md
@@ -22,11 +22,19 @@ into Rust bindings. This example registers one object,
`rust_stubgen.IntPair`
(`src/int_pair.cc`), and lets CMake regenerate `rust/src/generated/` after
every build.
-Every object is bound *opaquely*: Rust gets a `#[repr(C)]` wrapper that embeds
-only the parent, a reference type, `Deref`, the upcasts along the ancestor
-chain, and one accessor per reflected field that reads through the C ABI
-getter. The object's bytes are never reproduced, so the binding is correct for
-any registered type; construction goes through the registered global functions.
+Every object gets a `#[repr(C)]` wrapper, a reference type, `Deref`, and the
+upcasts along its ancestor chain. `IntPair` is plain data, so its reflected
+fields account for every byte and the binding is *complete*: the struct mirrors
+the fields at their real offsets and widths, a `const` assertion pins its size
+and alignment to the reflected facts, and a generated `new` allocates the
object
+in Rust. `main.rs` builds one that way, reads `pair.a` directly, and hands it
to
+a C++ function that reads it back.
+
+An object whose layout cannot be reproduced (a polymorphic one, say, with a
+vtable in front of the object header) is bound *opaquely* instead: the struct
+embeds only the parent, every field is read through an accessor that calls the
+C ABI getter, and construction stays on the C++ side.
+
A builtin parent such as `ffi.IntEnum` has no `<Leaf>Obj` in the crate; the
import section defines a header-only stand-in per builtin ancestor so the
derived type depth matches the registry.
@@ -56,6 +64,10 @@ open newtype:
// tvm-ffi-stubgen(enum): rust_stubgen.IntPair.kind -> PairKind(i32) {
Unordered=0, Ordered=1 }
```
-Two more are available: `field` names the Rust type of a field's accessor
-(`// tvm-ffi-stubgen(field): rust_stubgen.IntPair.a -> MyInt`) and `nullable`
-wraps it in `Option` (`// tvm-ffi-stubgen(nullable): rust_stubgen.IntPair.a`).
+Four more directives are available: `field` names the Rust type of a field
+(`// tvm-ffi-stubgen(field): rust_stubgen.IntPair.a -> MyInt`), `nullable`
+wraps it in `Option` (`// tvm-ffi-stubgen(nullable): rust_stubgen.IntPair.a`),
+`upcast` adds a conversion to a hand-written typed view
+(`// tvm-ffi-stubgen(upcast): rust_stubgen.IntPair -> MyView`), and
+`custom-new` names the generated allocator `from_complete_fields` when `new`
+is hand-written (`// tvm-ffi-stubgen(custom-new): rust_stubgen.IntPair`).
diff --git a/examples/rust_stubgen/rust/src/generated/rust_stubgen/mod.rs
b/examples/rust_stubgen/rust/src/generated/rust_stubgen/mod.rs
index 4e7f47e6..d1085be2 100644
--- a/examples/rust_stubgen/rust/src/generated/rust_stubgen/mod.rs
+++ b/examples/rust_stubgen/rust/src/generated/rust_stubgen/mod.rs
@@ -23,16 +23,14 @@
// tvm-ffi-stubgen(begin): import-section
use std::ops::Deref;
use tvm_ffi::Error;
-use tvm_ffi::FieldGetter;
use tvm_ffi::Object;
use tvm_ffi::ObjectArc;
-use tvm_ffi::ObjectCore;
use tvm_ffi::Result;
use tvm_ffi::VALUE_ERROR;
// tvm-ffi-stubgen(end)
-// The `kind` field is an integer on the C++ side; this directive gives it an
-// open integer newtype in Rust and makes the accessor return it.
+// The `kind` field is an integer on the C++ side; this directive types it as
an
+// open integer newtype in Rust.
// tvm-ffi-stubgen(enum): rust_stubgen.IntPair.kind -> PairKind(i32) {
Unordered=0, Ordered=1 }
// tvm-ffi-stubgen(begin): object/rust_stubgen.IntPair
@@ -61,14 +59,23 @@ impl TryFrom<i64> for PairKind {
}
}
+/// Complete: reflected fields fill [24, 48) exactly (alignment padding [44,
48)).
#[repr(C)]
#[derive(tvm_ffi::derive::Object)]
#[type_key = "rust_stubgen.IntPair"]
#[type_final]
pub struct IntPairObj {
base: Object,
+ pub a: i64,
+ pub b: i64,
+ pub kind: PairKind,
}
+const _: () = {
+ assert!(::core::mem::size_of::<IntPairObj>() == 48);
+ assert!(::core::mem::align_of::<IntPairObj>() == 8);
+};
+
#[repr(C)]
#[derive(tvm_ffi::derive::ObjectRef, Clone)]
pub struct IntPair {
@@ -83,17 +90,17 @@ impl Deref for IntPair {
}
impl IntPairObj {
- pub fn a(&self) -> Result<i64> {
- FieldGetter::new(Self::type_index(), "a")?.get(self)
- }
-
- pub fn b(&self) -> Result<i64> {
- FieldGetter::new(Self::type_index(), "b")?.get(self)
+ pub(crate) fn new(a: i64, b: i64, kind: PairKind) -> Self {
+ let base = Object::new();
+ Self { base, a, b, kind }
}
+}
- pub fn kind(&self) -> Result<PairKind> {
- let raw: i64 = FieldGetter::new(Self::type_index(),
"kind")?.get(self)?;
- PairKind::try_from(raw)
+impl IntPair {
+ /// Lossless complete-field allocation.
+ pub fn new(a: i64, b: i64, kind: PairKind) -> Self {
+ let obj = IntPairObj::new(a, b, kind);
+ Self { data: ObjectArc::new(obj) }
}
}
// tvm-ffi-stubgen(end)
diff --git a/examples/rust_stubgen/rust/src/main.rs
b/examples/rust_stubgen/rust/src/main.rs
index 27afeb3a..f49fadbe 100644
--- a/examples/rust_stubgen/rust/src/main.rs
+++ b/examples/rust_stubgen/rust/src/main.rs
@@ -37,20 +37,19 @@ fn lib_path() -> String {
fn main() -> Result<()> {
// Load the C++ library so `IntPair` is registered with the FFI type
registry.
- // Keep it alive for as long as the bindings are used.
+ // Keep it alive for as long as the binding is used.
let _lib = Module::load_from_file(lib_path())?;
- // The object is opaque to Rust: it is constructed by the registered C++
- // function and its fields are read through the reflection getters.
- let pair: IntPair = tvm_ffi::cached_global_func!("rust_stubgen.IntPair")
- .call_tuple((1i64, 2i64, i64::from(PairKind::Ordered.as_raw())))?
- .try_into()?;
- println!("a={} b={} kind={:?}", pair.a()?, pair.b()?, pair.kind()?);
- assert_eq!(pair.kind()?, PairKind::Ordered);
+ // The object has a reproducible layout: it is allocated in Rust and its
+ // fields are plain struct members, on both sides of the ABI.
+ let pair = IntPair::new(1, 2, PairKind::Ordered);
+ println!("a={} b={} kind={:?}", pair.a, pair.b, pair.kind);
+ assert_eq!(pair.kind, PairKind::Ordered);
let sum: i64 = tvm_ffi::cached_global_func!("rust_stubgen.IntPairSum")
.call_tuple((pair.clone(),))?
.try_into()?;
println!("sum={sum}");
+ assert_eq!(sum, 3);
Ok(())
}
diff --git a/examples/rust_stubgen/src/int_pair.cc
b/examples/rust_stubgen/src/int_pair.cc
index 3d7631b4..b758333d 100644
--- a/examples/rust_stubgen/src/int_pair.cc
+++ b/examples/rust_stubgen/src/int_pair.cc
@@ -29,9 +29,9 @@ namespace rust_stubgen {
namespace ffi = tvm::ffi;
// [object.begin]
-// A polymorphic object: the vtable in front of the object header means Rust
-// cannot mirror its bytes, so the generated binding reads every field through
-// the reflection getters and construction stays on the C++ side.
+// A plain data object: every byte is accounted for by a reflected field, so
the
+// generated binding mirrors the layout; Rust allocates it and reads the fields
+// directly, and the registered function below reads it back.
class IntPairObj : public ffi::Object {
public:
int64_t a;
@@ -39,16 +39,13 @@ class IntPairObj : public ffi::Object {
int32_t kind;
IntPairObj(int64_t a, int64_t b, int32_t kind) : a(a), b(b), kind(kind) {}
- virtual ~IntPairObj() = default;
- virtual int64_t Sum() const { return a + b; }
+ int64_t Sum() const { return a + b; }
TVM_FFI_DECLARE_OBJECT_INFO_FINAL("rust_stubgen.IntPair", IntPairObj,
ffi::Object);
};
class IntPair : public ffi::ObjectRef {
public:
- IntPair(int64_t a, int64_t b, int32_t kind) { data_ =
ffi::make_object<IntPairObj>(a, b, kind); }
-
TVM_FFI_DEFINE_OBJECT_REF_METHODS_NOTNULLABLE(IntPair, ffi::ObjectRef,
IntPairObj);
};
@@ -58,10 +55,7 @@ TVM_FFI_STATIC_INIT_BLOCK() {
.def_ro("a", &IntPairObj::a, "the first operand")
.def_ro("b", &IntPairObj::b, "the second operand")
.def_ro("kind", &IntPairObj::kind, "0 = unordered, 1 = ordered");
- refl::GlobalDef()
- .def("rust_stubgen.IntPair",
- [](int64_t a, int64_t b, int32_t kind) { return IntPair(a, b,
kind); })
- .def("rust_stubgen.IntPairSum", [](const IntPair& pair) { return
pair->Sum(); });
+ refl::GlobalDef().def("rust_stubgen.IntPairSum", [](const IntPair& pair) {
return pair->Sum(); });
}
// [object.end]
diff --git a/python/tvm_ffi/stub/layout.py b/python/tvm_ffi/stub/layout.py
index 798676a4..73c68bed 100644
--- a/python/tvm_ffi/stub/layout.py
+++ b/python/tvm_ffi/stub/layout.py
@@ -70,6 +70,7 @@ OpaqueReason: TypeAlias = Literal[
"uncovered-bytes",
"unrenderable-field",
"by-directive",
+ "no-mirror",
]
"""Why a type is opaque.
@@ -82,6 +83,8 @@ OpaqueReason: TypeAlias = Literal[
a vptr, ...).
- ``unrenderable-field``: the caller's predicate rejected a field.
- ``by-directive``: the caller vetoed a type whose layout is reproducible.
+- ``no-mirror``: the caller's target never reproduces this type's bytes
+ (a builtin its runtime owns); everything below it is ``parent-opaque``.
"""
@@ -175,6 +178,7 @@ def classify(
infos: Mapping[str, ObjectInfo],
*,
forced_opaque: AbstractSet[str] = frozenset(),
+ unmirrored: AbstractSet[str] = frozenset(),
field_renderable: Callable[[NamedTypeSchema], bool] | None = None,
) -> dict[str, Verdict]:
"""Classify every type in ``infos``, parents before children.
@@ -188,6 +192,9 @@ def classify(
Type keys vetoed by the caller (semantic blockers such as interned
identities). The veto only demotes a type that would otherwise be
complete; a type that is opaque for a layout reason keeps that reason.
+ unmirrored
+ Type keys whose bytes the target never reproduces (builtins its runtime
+ owns): opaque with reason ``no-mirror`` before any layout evidence is
weighed.
field_renderable
Predicate deciding whether a reflected field has a native mirror in the
target language. A rejected field demotes its type to opaque with
@@ -203,7 +210,9 @@ def classify(
raise KeyError(f"Ancestor {type_key!r} is not among the types to
classify")
info = infos[type_key]
parent = None if info.parent_type_key is None else
_classify(info.parent_type_key)
- verdicts[type_key] = _classify_one(info, parent, forced_opaque,
field_renderable)
+ verdicts[type_key] = _classify_one(
+ info, parent, forced_opaque, unmirrored, field_renderable
+ )
return verdicts[type_key]
for type_key in infos:
@@ -215,6 +224,7 @@ def _classify_one(
info: ObjectInfo,
parent: Verdict | None,
forced_opaque: AbstractSet[str],
+ unmirrored: AbstractSet[str],
field_renderable: Callable[[NamedTypeSchema], bool] | None,
) -> Verdict:
assert info.type_key is not None, "cannot classify an ObjectInfo without a
type key"
@@ -228,6 +238,10 @@ def _classify_one(
total_size=info.total_size,
is_final=info.is_final,
)
+ if info.type_key in unmirrored:
+ verdict.reason = "no-mirror"
+ verdict.detail = "its bytes are owned by the target's runtime and
never mirrored"
+ return verdict
outcome = _prove_layout(info, parent, verdict)
if outcome is None:
outcome = _apply_target_rules(info, forced_opaque, field_renderable)
diff --git a/python/tvm_ffi/stub/rust_generator/codegen.py
b/python/tvm_ffi/stub/rust_generator/codegen.py
index 26f2603a..78d778fb 100644
--- a/python/tvm_ffi/stub/rust_generator/codegen.py
+++ b/python/tvm_ffi/stub/rust_generator/codegen.py
@@ -14,45 +14,27 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Rust code generation for ``tvm-ffi-stubgen``: the opaque binding form.
-
-Every reflected object renders as a ``#[repr(C)]`` struct embedding only its
-parent, a reference wrapper, ``Deref``, the upcasts along the ancestor chain,
-and one accessor per reflected field that reads through the C ABI getter. The
-object's bytes are never reproduced. For ``tirx.IterVar`` deriving from
-``ir.PrimExprConvertible``::
-
- #[repr(C)]
- #[derive(tvm_ffi::derive::Object)]
- #[type_key = "tirx.IterVar"]
- #[type_final]
- pub struct IterVarObj {
- base: PrimExprConvertibleObj,
- }
-
- #[repr(C)]
- #[derive(tvm_ffi::derive::ObjectRef, Clone)]
- pub struct IterVar {
- data: ObjectArc<IterVarObj>,
- }
-
- impl Deref for IterVar { ... } // IterVar -> IterVarObj
- impl Deref for IterVarObj { ... } // IterVarObj ->
PrimExprConvertibleObj
-
- impl IterVarObj {
- pub fn dom(&self) -> Result<Option<Range>> {
- FieldGetter::new(Self::type_index(), "dom")?.get(self)
- }
- ...
- }
-
- tvm_ffi::impl_object_upcast!(IterVar => PrimExprConvertible);
-
-Construction and behaviour go through the registered global functions,
-hand-written outside the markers. A builtin parent (``ffi.IntEnum``, say) has
-no ``<Leaf>Obj`` in the crate: the import section defines a header-only
-stand-in per builtin ancestor, so ``derive(Object)`` computes the registry's
-``TYPE_DEPTH``.
+"""Rust code generation for ``tvm-ffi-stubgen``.
+
+Every reflected object gets a ``#[repr(C)]`` object struct, a reference
wrapper,
+``Deref``, and the upcasts along its ancestor chain. The struct's contents
follow
+the verdict of :mod:`tvm_ffi.stub.layout`:
+
+- *complete*: the struct mirrors every physical field at its real offset and
+ width, pinned by a ``const`` size/alignment assertion. ``<Leaf>Obj::new``
+ (crate-private) and the wrapper's ``new`` take every field root to leaf;
+ ``custom-new`` leaves the wrapper's ``new`` to hand-written code and names
+ the generated one ``from_complete_fields``.
+- *opaque*: the struct embeds only its parent, and each field is read through
+ the C ABI getter. Nothing allocates it.
+
+A field without a Rust mirror (``Optional<Any>``, ``Union``, ``void*``, ...)
+makes the type opaque and is read as ``Any``; an ``opaque`` directive vetoes a
+reproducible layout; a scalar width named by a directive is checked against the
+reflected field size. A builtin parent (``ffi.IntEnum``, say) has no
+``<Leaf>Obj`` in the crate: the import section defines a header-only stand-in
+per builtin ancestor so ``derive(Object)`` computes the registry's
+``TYPE_DEPTH``, and everything under such a parent stays opaque
(``no-mirror``).
"""
from __future__ import annotations
@@ -61,6 +43,8 @@ import dataclasses
from typing import TYPE_CHECKING
from .. import consts as C
+from ..layout import Verdict, classify
+from ..lib_state import object_info_from_type_key
from . import consts as C_RUST
from .utils import RustImports, builtin_mirror_name, render_rust_type,
rust_ident
@@ -72,6 +56,24 @@ if TYPE_CHECKING:
from .directives import EnumSpec
+def _call_lines(open_: str, items: list[str], close: str) -> list[str]:
+ """``open_ + items + close`` on one line, or one item per line when it
would overflow."""
+ line = f"{open_}{', '.join(items)}{close}"
+ if len(line) <= C_RUST.RUST_MAX_WIDTH:
+ return [line]
+ indent = open_[: len(open_) - len(open_.lstrip())]
+ return [open_, *[f"{indent} {item}," for item in items],
f"{indent}{close}"]
+
+
+def _check_width(target: str, field: NamedTypeSchema, rust_type: str, width:
int) -> None:
+ """Reject a directive whose scalar type does not match the reflected field
size."""
+ if field.size is not None and field.size != width:
+ raise ValueError(
+ f"Directive on `{target}` maps a {field.size}-byte field to
`{rust_type}` "
+ f"({width} bytes)"
+ )
+
+
@dataclasses.dataclass
class _ObjectRenderer:
"""Renders one ``object/<key>`` block into Rust source lines."""
@@ -100,14 +102,17 @@ class _ObjectRenderer:
# --- name resolution ---------------------------------------------------
- def _ty_render(self, origin: str) -> str | None:
+ def _resolve(self, origin: str, imports: RustImports) -> str | None:
"""Resolve a leaf origin to its in-scope Rust name (recording its
``use``), or ``None``."""
mapped = self.ty_map.get(origin)
if mapped is None:
if "." not in origin or origin.startswith("ctypes."):
return None
mapped = self._generated_type_path(origin)
- return self.imports.record(mapped)
+ return imports.record(mapped)
+
+ def _ty_render(self, origin: str) -> str | None:
+ return self._resolve(origin, self.imports)
def _generated_type_path(self, type_key: str) -> str:
"""Spell a generated type key from this file.
@@ -143,6 +148,95 @@ class _ObjectRenderer:
assert not any(self._generated(key) for key in chain), (self.type_key,
chain)
return self.imports.record_builtin_base(chain), False
+ # --- classification ----------------------------------------------------
+
+ def classify(self) -> Verdict:
+ """Classify this object with its ancestors, under the file's
directives."""
+ infos = {key: object_info_from_type_key(key) for key in
self.info.ancestors}
+ infos[self.type_key] = self.info
+ owner_of = {id(f): key for key, owner in infos.items() for f in
owner.fields}
+ scratch = RustImports()
+
+ def renderable(field: NamedTypeSchema) -> bool:
+ return self._field_mirror(owner_of[id(field)], field, scratch) is
not None
+
+ # Builtin ancestors are header-only stand-ins (`_base_type`): none
below is complete.
+ unmirrored = {
+ key
+ for key in self.info.ancestors
+ if key != C_RUST.RUST_ROOT_TYPE_KEY and not self._generated(key)
+ }
+ verdicts = classify(
+ infos,
+ forced_opaque=self.imports.directives.opaque,
+ unmirrored=unmirrored,
+ field_renderable=renderable,
+ )
+ return verdicts[self.type_key]
+
+ # --- field types -------------------------------------------------------
+
+ def _field_mirror(self, owner: str, field: NamedTypeSchema, imports:
RustImports) -> str | None:
+ """Render the ``#[repr(C)]`` mirror type of ``field``, or ``None``.
+
+ Directives win; then scalars by reflected width, ``Optional`` by C++
layout, else schema.
+ """
+ directives = self.imports.directives
+ target = f"{owner}.{field.name}"
+ enum = directives.enums.get(target)
+ if enum is not None:
+ _check_width(target, field, enum.repr,
C_RUST.RUST_SCALAR_WIDTHS[enum.repr])
+ return enum.name
+ override = directives.field_types.get(target)
+ if override is not None:
+ width = C_RUST.RUST_SCALAR_WIDTHS.get(override)
+ if width is not None:
+ _check_width(target, field, override, width)
+ mirror: str | None = imports.record(override) if "::" in override
else override
+ elif field.origin == "Optional":
+ mirror = self._optional_mirror(field, imports)
+ else:
+ narrowed = C_RUST.RUST_SCALAR_BY_SIZE.get((field.origin,
field.size))
+ mirror = narrowed or render_rust_type(field, lambda o:
self._resolve(o, imports))
+ if mirror is None:
+ return None
+ if target in directives.nullable and not mirror.startswith("Option<"):
+ if field.size not in (None, C_RUST.RUST_POINTER_SIZE):
+ raise ValueError(
+ f"`nullable` directive on `{target}`: the field is
{field.size} bytes, "
+ "not a pointer-sized object reference"
+ )
+ mirror = f"Option<{mirror}>"
+ return mirror
+
+ def _optional_mirror(self, field: NamedTypeSchema, imports: RustImports)
-> str | None:
+ """Mirror an ``Optional<T>`` field in place.
+
+ An object payload is a nullable pointer (``Option<T>``); any other
payload
+ is a 16-byte ``TVMFFIAny`` cell (``tvm_ffi::Optional<T>``).
``Optional<Any>``
+ and a size mismatch have no mirror.
+ """
+ (payload,) = field.args # TypeSchema's post_init enforces exactly one
argument.
+ if payload.origin == "Any":
+ return None
+ inner = render_rust_type(payload, lambda o: self._resolve(o, imports))
+ if inner is None:
+ return None
+ any_backed = (
+ payload.origin in C_RUST.RUST_ANY_BACKED_OPTIONAL_PAYLOADS
+ or payload.origin == "Optional"
+ )
+ expected = (
+ C_RUST.RUST_OPTIONAL_FIELD_SIZE
+ if any_backed
+ else C_RUST.RUST_OBJECT_OPTIONAL_FIELD_SIZE
+ )
+ if field.size not in (None, expected):
+ return None
+ if any_backed:
+ return f"{imports.record(C_RUST.RUST_OPTIONAL_PATH)}<{inner}>"
+ return f"Option<{inner}>"
+
# --- pieces ------------------------------------------------------------
def _accessor_lines(self, field: NamedTypeSchema) -> list[str]:
@@ -183,6 +277,7 @@ class _ObjectRenderer:
"""Render the open integer newtype an ``enum`` directive declares."""
error = self.imports.record("tvm_ffi::Error")
value_error = self.imports.record("tvm_ffi::VALUE_ERROR")
+ result = self.imports.record("tvm_ffi::Result")
return [
"#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]",
"#[repr(transparent)]",
@@ -201,7 +296,7 @@ class _ObjectRenderer:
"",
f"impl TryFrom<i64> for {spec.name} {{",
f" type Error = {error};",
- " fn try_from(value: i64) -> Result<Self> {",
+ f" fn try_from(value: i64) -> {result}<Self> {{",
f" {spec.repr}::try_from(value).map(Self).map_err(|_| {{",
f' {error}::new({value_error}, &format!("{spec.name}
value {{value}} does not fit '
f'{spec.repr}"), "")',
@@ -221,25 +316,151 @@ class _ObjectRenderer:
]
def _upcast_lines(self) -> list[str]:
- """``impl_object_upcast!`` from the wrapper to every ancestor's
wrapper."""
+ """``impl_object_upcast!`` to every ancestor's wrapper, then the
``upcast`` directives."""
targets = [
self.imports.record(self._generated_type_path(key))
for key in self.info.ancestors
if self._generated(key)
]
+ for view in self.imports.directives.upcasts.get(self.type_key, []):
+ targets.append(self.imports.record(view) if "::" in view else view)
if not targets:
return []
pairs = ", ".join(f"{self.leaf} => {target}" for target in targets)
return [f"tvm_ffi::impl_object_upcast!({pairs});"]
+ def _struct_lines(self, verdict: Verdict, base: str) -> list[str]:
+ """Render the object struct: every field when complete, the parent
alone when opaque."""
+ header = [
+ "#[repr(C)]",
+ "#[derive(tvm_ffi::derive::Object)]",
+ f'#[type_key = "{self.type_key}"]',
+ *(["#[type_final]"] if self.info.is_final else []),
+ f"pub struct {self.obj_struct} {{",
+ f" base: {base},", # a reflected `base` field becomes `base_`
(`rust_ident`)
+ ]
+ if not verdict.is_complete:
+ return [
+ f"/// Opaque: {verdict.detail}. Fields are read through the C
ABI getters.",
+ *header,
+ "}",
+ ]
+ members = []
+ for field in sorted(self.info.fields, key=lambda f: f.offset or 0):
+ mirror = self._field_mirror(self.type_key, field, self.imports)
+ assert mirror is not None # the verdict already ran the
renderability check
+ members.append(f" pub {rust_ident(field.name)}: {mirror},")
+ return [
+ f"/// Complete: {verdict.detail}.",
+ *header,
+ *members,
+ "}",
+ "",
+ "const _: () = {",
+ f" assert!(::core::mem::size_of::<{self.obj_struct}>() ==
{verdict.total_size});",
+ f" assert!(::core::mem::align_of::<{self.obj_struct}>() ==
{verdict.alignment});",
+ "};",
+ ]
+
+ # --- allocators --------------------------------------------------------
+
+ def _allocator_params(self, key: str, info: ObjectInfo) -> list[tuple[str,
str]]:
+ """``(field, type)`` of every physical field root to leaf, as
``<key>Obj::new`` takes."""
+ parent = info.parent_type_key
+ inherited: list[tuple[str, str]] = []
+ if parent is not None and self._generated(parent):
+ inherited = self._allocator_params(parent,
object_info_from_type_key(parent))
+ return self._level_params(key, info, inherited)
+
+ def _level_params(
+ self, key: str, info: ObjectInfo, inherited: list[tuple[str, str]]
+ ) -> list[tuple[str, str]]:
+ """Extend the parent's parameters with ``key``'s own fields by offset.
+
+ A ``field`` directive on an inherited field narrows that parameter; the
+ body hands it to the parent with ``.into()``.
+ """
+ params = [(name, self._narrowed(key, name, rust_type)) for name,
rust_type in inherited]
+ for field in sorted(info.fields, key=lambda f: f.offset or 0):
+ mirror = self._field_mirror(key, field, self.imports)
+ assert mirror is not None # complete: every field along the chain
has a mirror
+ params.append((field.name, mirror))
+ return params
+
+ def _narrowed(self, key: str, field_name: str, rust_type: str) -> str:
+ override =
self.imports.directives.field_types.get(f"{key}.{field_name}")
+ if override is None:
+ return rust_type
+ return self.imports.record(override) if "::" in override else override
+
+ def _fn_lines(
+ self, head: str, params: list[tuple[str, str]], call: tuple[str,
list[str]], result: str
+ ) -> list[str]:
+ """Render ``<head>(<params>) -> Self { let <call>; <result> }`` inside
an ``impl`` block."""
+ plist = [f"{rust_ident(name)}: {rust_type}" for name, rust_type in
params]
+ binding, args = call
+ return [
+ *_call_lines(f" {head}(", plist, ") -> Self {"),
+ *_call_lines(f" let {binding}(", args, ");"),
+ f" {result}",
+ " }",
+ ]
+
+ def _allocator_sections(self, base: str, has_parent: bool) ->
list[list[str]]:
+ """``<Leaf>Obj::new`` and the wrapper's ``new``.
+
+ ``custom-new`` names the wrapper's allocator ``from_complete_fields``
instead.
+ """
+ inherited: list[tuple[str, str]] = []
+ if has_parent:
+ parent = self.info.parent_type_key
+ assert parent is not None
+ inherited = self._allocator_params(parent,
object_info_from_type_key(parent))
+ params = self._level_params(self.type_key, self.info, inherited)
+ to_parent = [
+ f"{rust_ident(name)}.into()" if rust_type != parent_type else
rust_ident(name)
+ for (name, rust_type), (_, parent_type) in zip(params, inherited)
+ ]
+ own = [rust_ident(f.name) for f in sorted(self.info.fields, key=lambda
f: f.offset or 0)]
+ forward = [rust_ident(name) for name, _ in params]
+ sections = [
+ [
+ f"impl {self.obj_struct} {{",
+ *self._fn_lines(
+ "pub(crate) fn new",
+ params,
+ (f"base = {base}::new", to_parent),
+ f"Self {{ {', '.join(['base', *own])} }}",
+ ),
+ "}",
+ ]
+ ]
+ custom = self.type_key in self.imports.directives.custom_new
+ sections.append(
+ [
+ f"impl {self.leaf} {{",
+ " /// Lossless complete-field allocation.",
+ *self._fn_lines(
+ "pub fn from_complete_fields" if custom else "pub fn new",
+ params,
+ (f"obj = {self.obj_struct}::new", forward),
+ "Self { data: ObjectArc::new(obj) }",
+ ),
+ "}",
+ ]
+ )
+ return sections
+
def body(self) -> list[str]:
"""Build the Rust source lines for the object."""
+ verdict = self.classify()
# Derive macros are spelled by full path: their leaves collide with
`Object` / `ObjectRef`.
self.imports.record("std::ops::Deref")
self.imports.record("tvm_ffi::ObjectArc")
base, has_parent = self._base_type()
fields = self.info.fields
- if fields:
+ has_accessors = bool(fields) and not verdict.is_complete
+ if has_accessors:
self.imports.record("tvm_ffi::ObjectCore") # `Self::type_index()`
self.imports.record("tvm_ffi::FieldGetter")
self.imports.record("tvm_ffi::Result")
@@ -251,22 +472,13 @@ class _ObjectRenderer:
for f in fields
if f"{self.type_key}.{f.name}" in enums
]
- sections.append(
- [
- "#[repr(C)]",
- "#[derive(tvm_ffi::derive::Object)]",
- f'#[type_key = "{self.type_key}"]',
- *(["#[type_final]"] if self.info.is_final else []),
- f"pub struct {self.obj_struct} {{",
- f" base: {base},",
- "}",
- ]
- )
+ sections.append(self._struct_lines(verdict, base))
sections.append(
[
"#[repr(C)]",
"#[derive(tvm_ffi::derive::ObjectRef, Clone)]",
f"pub struct {self.leaf} {{",
+ # a reflected `data` field becomes `data_` (`rust_ident`)
f" data: ObjectArc<{self.obj_struct}>,",
"}",
]
@@ -274,7 +486,7 @@ class _ObjectRenderer:
sections.append(self._deref_lines(self.leaf, self.obj_struct, "data"))
if has_parent:
sections.append(self._deref_lines(self.obj_struct, base, "base"))
- if fields:
+ if has_accessors:
accessors: list[str] = []
for i, field in enumerate(fields):
if i:
@@ -287,6 +499,8 @@ class _ObjectRenderer:
"}",
]
)
+ elif verdict.is_complete:
+ sections += self._allocator_sections(base, has_parent)
upcasts = self._upcast_lines()
if upcasts:
sections.append(upcasts)
@@ -306,7 +520,7 @@ def generate_rust_object(
opt: Options,
obj_info: ObjectInfo,
) -> None:
- """Emit the opaque Rust binding of ``obj_info`` into an ``object/<key>``
block."""
+ """Emit the Rust binding of ``obj_info`` into an ``object/<key>`` block."""
assert len(code.lines) >= 2
assert isinstance(obj_info.type_key, str)
renderer = _ObjectRenderer(
diff --git a/python/tvm_ffi/stub/rust_generator/consts.py
b/python/tvm_ffi/stub/rust_generator/consts.py
index 218def67..0636cbfb 100644
--- a/python/tvm_ffi/stub/rust_generator/consts.py
+++ b/python/tvm_ffi/stub/rust_generator/consts.py
@@ -19,7 +19,9 @@
from __future__ import annotations
#: One-line directives the Rust backend consumes.
-RUST_DIRECTIVE_KINDS = frozenset({"import-object", "field", "nullable",
"enum"})
+RUST_DIRECTIVE_KINDS = frozenset(
+ {"import-object", "field", "nullable", "enum", "opaque", "upcast",
"custom-new"}
+)
#: Default FFI-origin -> Rust-type map; ``::`` paths get a ``use``, bare names
do not.
RUST_TY_MAP_DEFAULTS = {
@@ -54,6 +56,44 @@ RUST_TY_MAP_DEFAULTS = {
#: Origins without a crate mirror; such a field is read as ``tvm_ffi::Any``.
RUST_UNSUPPORTED_ORIGINS = frozenset({"Dict", "List", "Union", "tuple"})
+#: Mirror scalar by ``(origin, reflected size)``: the schema erases widths and
signedness.
+RUST_SCALAR_BY_SIZE = {
+ ("int", 1): "i8",
+ ("int", 2): "i16",
+ ("int", 4): "i32",
+ ("int", 8): "i64",
+ ("float", 4): "f32",
+ ("float", 8): "f64",
+}
+
+#: Byte width of the scalars a ``field`` / ``enum`` directive may name;
checked against the field.
+RUST_SCALAR_WIDTHS = {
+ "i8": 1,
+ "u8": 1,
+ "bool": 1,
+ "i16": 2,
+ "u16": 2,
+ "i32": 4,
+ "u32": 4,
+ "f32": 4,
+ "i64": 8,
+ "u64": 8,
+ "f64": 8,
+}
+
+#: Size of an object reference field; ``nullable`` may only wrap those.
+RUST_POINTER_SIZE = 8
+
+#: C++ ``Optional<T>`` is a 16-byte ``TVMFFIAny`` cell, or a nullable pointer
for object payloads.
+RUST_OPTIONAL_PATH = "tvm_ffi::Optional"
+RUST_OPTIONAL_FIELD_SIZE = 16
+RUST_OBJECT_OPTIONAL_FIELD_SIZE = 8
+
+#: ``Optional`` payloads kept as the 16-byte cell (a nested ``Optional`` too);
the size is checked.
+RUST_ANY_BACKED_OPTIONAL_PAYLOADS = frozenset(
+ {"int", "float", "bool", "Device", "dtype", "DataType", "str", "bytes"}
+)
+
#: ``use``-path rewrites: builtin ``ffi.*`` type keys live at the crate root.
RUST_MOD_MAP = {
"ffi": "tvm_ffi",
@@ -71,3 +111,11 @@ RUST_KEYWORDS = frozenset(
"unsized virtual yield".split()
)
RUST_NOT_RAW_IDENTIFIERS = frozenset({"self", "Self", "super", "crate"})
+
+#: Member names the generated structs use themselves: ``base`` is the parent
slot of every object
+#: struct, ``data`` the wrapper's ``ObjectArc``. A reflected field with one of
these names would
+#: collide (or shadow through ``Deref``), so ``rust_ident`` spells it
``base_`` / ``data_``.
+RUST_RESERVED_MEMBERS = frozenset({"base", "data"})
+
+#: ``rustfmt``'s default ``max_width``; a wider allocator signature wraps one
parameter per line.
+RUST_MAX_WIDTH = 100
diff --git a/python/tvm_ffi/stub/rust_generator/directives.py
b/python/tvm_ffi/stub/rust_generator/directives.py
index 388ffb5f..185b9065 100644
--- a/python/tvm_ffi/stub/rust_generator/directives.py
+++ b/python/tvm_ffi/stub/rust_generator/directives.py
@@ -16,15 +16,22 @@
# under the License.
"""The Rust backend's one-line directives: payload grammar and per-file
storage.
-All three address one reflected field as ``<type_key>.<field>``::
+Three address one reflected field as ``<type_key>.<field>``, three address a
type::
// tvm-ffi-stubgen(field): tirx.Add.a -> PrimExpr
// tvm-ffi-stubgen(nullable): ir.Expr.span
// tvm-ffi-stubgen(enum): tirx.For.kind -> ForKind(i32) { Serial=0,
Parallel=1 }
-
-``field`` sets the accessor's Rust type (a name in scope, or a ``::`` path to
-``use``); ``nullable`` wraps it in ``Option``; ``enum`` declares an open
integer
-newtype the accessor returns.
+ // tvm-ffi-stubgen(opaque): ir.SourceName
+ // tvm-ffi-stubgen(upcast): tirx.Add -> PrimExpr
+ // tvm-ffi-stubgen(custom-new): tirx.Add
+
+``field`` sets the field's Rust type (a name in scope, or a ``::`` path to
+``use``); on a field inherited from an ancestor it narrows the allocator
+parameter instead. ``nullable`` wraps it in ``Option``; ``enum`` declares an
open
+integer newtype for it; ``opaque`` keeps a type opaque even when its layout is
+reproducible. ``upcast`` adds a typed view outside the ancestor chain;
+``custom-new`` says the wrapper's ``new`` is hand-written; the generated one is
+named ``from_complete_fields`` instead.
"""
from __future__ import annotations
@@ -50,22 +57,32 @@ class EnumSpec:
@dataclasses.dataclass
class Directives:
- """The Rust directives of one file, keyed by ``<type_key>.<field>``."""
+ """The Rust directives of one file, keyed by ``<type_key>.<field>`` or
``<type_key>``."""
field_types: dict[str, str] = dataclasses.field(default_factory=dict)
nullable: set[str] = dataclasses.field(default_factory=set)
enums: dict[str, EnumSpec] = dataclasses.field(default_factory=dict)
+ opaque: set[str] = dataclasses.field(default_factory=set)
+ upcasts: dict[str, list[str]] = dataclasses.field(default_factory=dict)
+ custom_new: set[str] = dataclasses.field(default_factory=set)
def add(self, name: str, payload: str, lineno: int) -> None:
"""Parse and store one directive; raise ``ValueError`` on a malformed
payload."""
if name == "field":
- target, rust_type = _split_arrow(name, payload, lineno)
- self.field_types[target] = rust_type
+ lhs, rust_type = _split_arrow(name, payload, lineno,
"<type_key>.<field> -> <RustType>")
+ self.field_types[_field_target(name, lhs, lineno)] = rust_type
elif name == "nullable":
self.nullable.add(_field_target(name, payload, lineno))
elif name == "enum":
target, spec = _parse_enum(payload, lineno)
self.enums[target] = spec
+ elif name == "opaque":
+ self.opaque.add(_type_target(name, payload, lineno))
+ elif name == "upcast":
+ lhs, rust_type = _split_arrow(name, payload, lineno, "<type_key>
-> <RustType>")
+ self.upcasts.setdefault(_type_target(name, lhs, lineno),
[]).append(rust_type)
+ elif name == "custom-new":
+ self.custom_new.add(_type_target(name, payload, lineno))
else:
raise ValueError(f"Unknown directive `{name}` at line {lineno}")
@@ -74,6 +91,14 @@ def _invalid(name: str, lineno: int, expected: str) ->
ValueError:
return ValueError(f"Invalid `{name}` directive at line {lineno}. Expected
`{expected}`")
+def _type_target(name: str, text: str, lineno: int) -> str:
+ """Validate a ``<type_key>`` reference."""
+ target = text.strip()
+ if not target or " " in target:
+ raise _invalid(name, lineno, "<type_key>")
+ return target
+
+
def _field_target(name: str, text: str, lineno: int) -> str:
"""Validate a ``<type_key>.<field>`` reference."""
target = text.strip()
@@ -82,12 +107,12 @@ def _field_target(name: str, text: str, lineno: int) ->
str:
return target
-def _split_arrow(name: str, payload: str, lineno: int) -> tuple[str, str]:
- """Split ``<type_key>.<field> -> <rust type>``."""
+def _split_arrow(name: str, payload: str, lineno: int, expected: str) ->
tuple[str, str]:
+ """Split ``<target> -> <rust type>``; the caller validates the target."""
lhs, arrow, rhs = payload.partition("->")
if not arrow or not rhs.strip():
- raise _invalid(name, lineno, "<type_key>.<field> -> <RustType>")
- return _field_target(name, lhs, lineno), rhs.strip()
+ raise _invalid(name, lineno, expected)
+ return lhs, rhs.strip()
def _parse_enum(payload: str, lineno: int) -> tuple[str, EnumSpec]:
diff --git a/python/tvm_ffi/stub/rust_generator/utils.py
b/python/tvm_ffi/stub/rust_generator/utils.py
index c60f52d7..27510efd 100644
--- a/python/tvm_ffi/stub/rust_generator/utils.py
+++ b/python/tvm_ffi/stub/rust_generator/utils.py
@@ -133,9 +133,13 @@ def _generic(base: str | None, *params: str | None) -> str
| None:
def rust_ident(name: str) -> str:
- """Spell a reflected field name in Rust: drop the C++ trailing underscore,
escape keywords."""
+ """Spell a reflected field name in Rust: drop the C++ trailing underscore,
escape collisions.
+
+ Keywords become raw identifiers; the four that cannot, and the names the
+ generated structs use themselves (``base``, ``data``), get a trailing
underscore.
+ """
name = name.rstrip("_") or name
- if name in C.RUST_NOT_RAW_IDENTIFIERS:
+ if name in C.RUST_NOT_RAW_IDENTIFIERS or name in C.RUST_RESERVED_MEMBERS:
return f"{name}_"
if name in C.RUST_KEYWORDS:
return f"r#{name}"
diff --git a/tests/python/test_stub_layout.py b/tests/python/test_stub_layout.py
index e0a6edcb..7e4379f5 100644
--- a/tests/python/test_stub_layout.py
+++ b/tests/python/test_stub_layout.py
@@ -239,6 +239,16 @@ def test_children_of_any_opaque_parent_are_parent_opaque()
-> None:
assert "by-directive" in verdicts["t.Child"].detail
+def test_unmirrored_type_and_its_descendants_are_opaque() -> None:
+ child = _info("t.Child", parent="t.Base", total_size=48,
fields=(_field("x", 40, 8),))
+ infos = {"ffi.Object": ROOT, "t.Base": BASE, "t.Child": child}
+ verdicts = classify(infos, unmirrored={"t.Base"})
+ assert verdicts["t.Base"].reason == "no-mirror"
+ assert verdicts["t.Base"].own_bytes is None # unconditional: no layout
evidence is weighed
+ assert verdicts["t.Child"].reason == "parent-opaque"
+ assert "no-mirror" in verdicts["t.Child"].detail
+
+
def test_order_does_not_matter_but_ancestors_must_be_present() -> None:
child = _info("t.Child", parent="t.Base", total_size=48,
fields=(_field("x", 40, 8),))
verdicts = classify({"t.Child": child, "t.Base": BASE, "ffi.Object": ROOT})
diff --git a/tests/python/test_stubgen_rust.py
b/tests/python/test_stubgen_rust.py
index 9f2e0fdf..446e1ade 100644
--- a/tests/python/test_stubgen_rust.py
+++ b/tests/python/test_stubgen_rust.py
@@ -14,11 +14,12 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-"""Tests for the Rust backend of ``tvm-ffi-stubgen``: opaque bindings."""
+"""Tests for the Rust backend of ``tvm-ffi-stubgen``: complete and opaque
bindings."""
from __future__ import annotations
import re
+from collections.abc import Iterator
from pathlib import Path
import pytest
@@ -29,6 +30,8 @@ from tvm_ffi.stub import consts as C
from tvm_ffi.stub.cli import _stage_3
from tvm_ffi.stub.file_utils import CodeBlock, FileInfo
from tvm_ffi.stub.generator import get_generator
+from tvm_ffi.stub.lib_state import object_info_from_type_key
+from tvm_ffi.stub.rust_generator import codegen
from tvm_ffi.stub.rust_generator import consts as RC
from tvm_ffi.stub.rust_generator.codegen import (
finalize_rust_module_tree,
@@ -41,28 +44,70 @@ from tvm_ffi.stub.rust_generator.utils import RustImports,
RustUse, render_rust_
from tvm_ffi.stub.utils import InitConfig, NamedTypeSchema, ObjectInfo, Options
RUST = get_generator("rust")
+HEADER = 24 # sizeof(TVMFFIObject)
+
+
+def _field(
+ name: str,
+ schema: str | TypeSchema,
+ offset: int | None = None,
+ size: int | None = None,
+ alignment: int | None = None,
+) -> NamedTypeSchema:
+ if isinstance(schema, str):
+ schema = TypeSchema(schema)
+ if alignment is None and size is not None:
+ alignment = min(size, 8) # a 16-byte `TVMFFIAny` cell is 8-aligned
+ return NamedTypeSchema(name, schema, offset=offset, size=size,
alignment=alignment)
def _info(
type_key: str,
- fields: tuple[tuple[str, TypeSchema], ...] = (),
+ fields: tuple[NamedTypeSchema, ...] = (),
*,
parent: str | None = "ffi.Object",
ancestors: list[str] | None = None,
is_final: bool | None = None,
+ total_size: int | None = None,
) -> ObjectInfo:
if ancestors is None:
ancestors = ["ffi.Object"] if parent in (None, "ffi.Object") else
["ffi.Object", parent]
return ObjectInfo(
- fields=[NamedTypeSchema(name, schema) for name, schema in fields],
+ fields=list(fields),
methods=[],
type_key=type_key,
parent_type_key=parent,
ancestors=ancestors,
is_final=is_final,
+ total_size=total_size,
)
+#: Ancestors the tests define with byte facts; anything else a test names but
+#: does not define resolves to a type without metadata of its own.
+_SYNTHETIC: dict[str, ObjectInfo] = {}
+
+
+def _register(*infos: ObjectInfo) -> None:
+ for info in infos:
+ assert info.type_key is not None
+ _SYNTHETIC[info.type_key] = info
+
+
[email protected](autouse=True)
+def _synthetic_registry(monkeypatch: pytest.MonkeyPatch) -> Iterator[None]:
+ def lookup(type_key: str) -> ObjectInfo:
+ if type_key in _SYNTHETIC:
+ return _SYNTHETIC[type_key]
+ if type_key.startswith(("ffi.", "testing.")):
+ return object_info_from_type_key(type_key)
+ return _info(type_key, total_size=None)
+
+ monkeypatch.setattr(codegen, "object_info_from_type_key", lookup)
+ yield
+ _SYNTHETIC.clear()
+
+
def _object_block(type_key: str) -> CodeBlock:
return CodeBlock(
kind="object",
@@ -86,6 +131,9 @@ def _uses(imports: RustImports) -> set[str]:
return {item.path for item in imports.items}
+NO_METADATA = "/// Opaque: no metadata of its own: total_size is unknown.
Fields are read through the C ABI getters."
+
+
# ---------------------------------------------------------------------------
# `use` modelling and type rendering
# ---------------------------------------------------------------------------
@@ -154,6 +202,8 @@ def test_rust_ident() -> None:
assert rust_ident("type") == "r#type"
assert rust_ident("self") == "self_"
assert rust_ident("crate") == "crate_"
+ assert rust_ident("base") == "base_" # the parent slot of every generated
object struct
+ assert rust_ident("data") == "data_" # the wrapper's `ObjectArc` member
# ---------------------------------------------------------------------------
@@ -167,12 +217,19 @@ def test_directives_parse() -> None:
directives.add("nullable", "ir.Expr.span", 2)
directives.add("enum", "tirx.For.kind -> ForKind(i32) { Serial=0, Parallel
= 1 }", 3)
directives.add("enum", "tirx.For.mode -> Mode(u8)", 4)
+ directives.add("opaque", " ir.SourceName ", 5)
+ directives.add("upcast", "tirx.Add -> PrimExpr", 6)
+ directives.add("upcast", "tirx.Add -> crate::typed::TypedExpr", 7)
+ directives.add("custom-new", " tirx.Add ", 8)
assert directives.field_types == {"tirx.Add.a": "PrimExpr"}
assert directives.nullable == {"ir.Expr.span"}
assert directives.enums == {
"tirx.For.kind": EnumSpec("ForKind", "i32", (("Serial", 0),
("Parallel", 1))),
"tirx.For.mode": EnumSpec("Mode", "u8", ()),
}
+ assert directives.opaque == {"ir.SourceName"}
+ assert directives.upcasts == {"tirx.Add": ["PrimExpr",
"crate::typed::TypedExpr"]}
+ assert directives.custom_new == {"tirx.Add"}
@pytest.mark.parametrize(
@@ -185,7 +242,11 @@ def test_directives_parse() -> None:
("enum", "tirx.For.kind -> ForKind", "Name(i32)"),
("enum", "tirx.For.kind -> ForKind(i128)", "Name(i32)"),
("enum", "tirx.For.kind -> ForKind(i32) { Serial }", "Name(i32)"),
- ("upcast", "tirx.Add -> PrimExpr", "Unknown directive"),
+ ("opaque", "ir.SourceName ir.Source", "<type_key>"),
+ ("upcast", "tirx.Add", "-> <RustType>"),
+ ("upcast", "tirx.Add PrimExpr -> PrimExpr", "<type_key>"),
+ ("custom-new", "", "<type_key>"),
+ ("typed-view", "tirx.Add -> PrimExpr", "Unknown directive"),
],
)
def test_directives_reject_malformed(name: str, payload: str, expected: str)
-> None:
@@ -195,7 +256,15 @@ def test_directives_reject_malformed(name: str, payload:
str, expected: str) ->
def test_generator_declares_its_directives_and_records_imports() -> None:
- assert RUST.directive_kinds == {"import-object", "field", "nullable",
"enum"}
+ assert RUST.directive_kinds == {
+ "import-object",
+ "field",
+ "nullable",
+ "enum",
+ "opaque",
+ "upcast",
+ "custom-new",
+ }
imports = RUST.new_imports()
RUST.add_directive(imports, "import-object",
"tvm_ffi.libinfo.Foo;False;_Foo", 1)
RUST.add_directive(imports, "nullable", "demo.Node.span", 2)
@@ -204,51 +273,52 @@ def
test_generator_declares_its_directives_and_records_imports() -> None:
# ---------------------------------------------------------------------------
-# Object rendering
+# Opaque rendering (no byte facts: the layout cannot be proven)
# ---------------------------------------------------------------------------
-ROOT_EXPECTED = """\
+ROOT_EXPECTED = f"""\
+{NO_METADATA}
#[repr(C)]
#[derive(tvm_ffi::derive::Object)]
#[type_key = "demo.Pair"]
-pub struct PairObj {
+pub struct PairObj {{
base: Object,
-}
+}}
#[repr(C)]
#[derive(tvm_ffi::derive::ObjectRef, Clone)]
-pub struct Pair {
+pub struct Pair {{
data: ObjectArc<PairObj>,
-}
+}}
-impl Deref for Pair {
+impl Deref for Pair {{
type Target = PairObj;
- fn deref(&self) -> &PairObj {
+ fn deref(&self) -> &PairObj {{
&self.data
- }
-}
+ }}
+}}
-impl PairObj {
- pub fn a(&self) -> Result<i64> {
+impl PairObj {{
+ pub fn a(&self) -> Result<i64> {{
FieldGetter::new(Self::type_index(), "a")?.get(self)
- }
+ }}
- pub fn tag(&self) -> Result<Option<String>> {
+ pub fn tag(&self) -> Result<Option<String>> {{
FieldGetter::new(Self::type_index(), "tag")?.get(self)
- }
+ }}
- pub fn items(&self) -> Result<Array<Any>> {
+ pub fn items(&self) -> Result<Array<Any>> {{
FieldGetter::new(Self::type_index(), "items")?.get(self)
- }
+ }}
- pub fn owner(&self) -> Result<ObjectRef> {
+ pub fn owner(&self) -> Result<ObjectRef> {{
FieldGetter::new(Self::type_index(), "owner")?.get(self)
- }
+ }}
- pub fn r#type(&self) -> Result<Any> {
+ pub fn r#type(&self) -> Result<Any> {{
FieldGetter::new(Self::type_index(), "type")?.get_any(self)
- }
-}"""
+ }}
+}}"""
def test_render_root_object() -> None:
@@ -256,11 +326,11 @@ def test_render_root_object() -> None:
info = _info(
"demo.Pair",
(
- ("a", TypeSchema("int")),
- ("tag", TypeSchema("Optional", (TypeSchema("str"),))),
- ("items", TypeSchema("Array")),
- ("owner", TypeSchema("Object")),
- ("type", TypeSchema("Union", (TypeSchema("int"),
TypeSchema("str")))),
+ _field("a", "int"),
+ _field("tag", TypeSchema("Optional", (TypeSchema("str"),))),
+ _field("items", "Array"),
+ _field("owner", "Object"),
+ _field("type", TypeSchema("Union", (TypeSchema("int"),
TypeSchema("str")))),
),
)
text, imports = _render(info)
@@ -290,7 +360,7 @@ def test_render_derived_object_same_module() -> None:
"""A generated parent is embedded, dereferenced to, and upcast to along
the chain."""
info = _info(
"demo.Add",
- (("a", TypeSchema("demo.Expr")),),
+ (_field("a", "demo.Expr"),),
parent="demo.Expr",
ancestors=["ffi.Object", "demo.BaseExpr", "demo.Expr"],
is_final=True,
@@ -317,7 +387,7 @@ def test_render_object_under_builtin_parent() -> None:
"""A builtin parent is embedded via header-only stand-ins so `TYPE_DEPTH`
matches the registry."""
info = _info(
"demo.Color",
- (("value", TypeSchema("int")),),
+ (_field("value", "int"),),
parent="ffi.IntEnum",
ancestors=["ffi.Object", "ffi.Enum", "ffi.IntEnum"],
)
@@ -401,82 +471,83 @@ def test_import_section_defines_builtin_mirrors_once() ->
None:
assert "\n".join(block.lines[1:-1]) == BUILTIN_MIRRORS_EXPECTED
-ITER_VAR_EXPECTED = """\
+ITER_VAR_EXPECTED = f"""\
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
#[repr(transparent)]
pub struct IterVarType(i32);
#[allow(non_upper_case_globals)]
-impl IterVarType {
+impl IterVarType {{
pub const kDataPar: Self = Self(0);
pub const kThreadIndex: Self = Self(1);
- pub const fn from_raw(value: i32) -> Self {
+ pub const fn from_raw(value: i32) -> Self {{
Self(value)
- }
- pub const fn as_raw(self) -> i32 {
+ }}
+ pub const fn as_raw(self) -> i32 {{
self.0
- }
-}
+ }}
+}}
-impl TryFrom<i64> for IterVarType {
+impl TryFrom<i64> for IterVarType {{
type Error = Error;
- fn try_from(value: i64) -> Result<Self> {
- i32::try_from(value).map(Self).map_err(|_| {
- Error::new(VALUE_ERROR, &format!("IterVarType value {value} does
not fit i32"), "")
- })
- }
-}
-
+ fn try_from(value: i64) -> Result<Self> {{
+ i32::try_from(value).map(Self).map_err(|_| {{
+ Error::new(VALUE_ERROR, &format!("IterVarType value {{value}} does
not fit i32"), "")
+ }})
+ }}
+}}
+
+{NO_METADATA}
#[repr(C)]
#[derive(tvm_ffi::derive::Object)]
#[type_key = "tirx.IterVar"]
#[type_final]
-pub struct IterVarObj {
+pub struct IterVarObj {{
base: PrimExprConvertibleObj,
-}
+}}
#[repr(C)]
#[derive(tvm_ffi::derive::ObjectRef, Clone)]
-pub struct IterVar {
+pub struct IterVar {{
data: ObjectArc<IterVarObj>,
-}
+}}
-impl Deref for IterVar {
+impl Deref for IterVar {{
type Target = IterVarObj;
- fn deref(&self) -> &IterVarObj {
+ fn deref(&self) -> &IterVarObj {{
&self.data
- }
-}
+ }}
+}}
-impl Deref for IterVarObj {
+impl Deref for IterVarObj {{
type Target = PrimExprConvertibleObj;
- fn deref(&self) -> &PrimExprConvertibleObj {
+ fn deref(&self) -> &PrimExprConvertibleObj {{
&self.base
- }
-}
+ }}
+}}
-impl IterVarObj {
- pub fn dom(&self) -> Result<Option<Range>> {
+impl IterVarObj {{
+ pub fn dom(&self) -> Result<Option<Range>> {{
FieldGetter::new(Self::type_index(), "dom")?.get(self)
- }
+ }}
- pub fn var(&self) -> Result<PrimVar> {
+ pub fn var(&self) -> Result<PrimVar> {{
FieldGetter::new(Self::type_index(), "var")?.get(self)
- }
+ }}
- pub fn iter_type(&self) -> Result<IterVarType> {
+ pub fn iter_type(&self) -> Result<IterVarType> {{
let raw: i64 = FieldGetter::new(Self::type_index(),
"iter_type")?.get(self)?;
IterVarType::try_from(raw)
- }
+ }}
- pub fn thread_tag(&self) -> Result<String> {
+ pub fn thread_tag(&self) -> Result<String> {{
FieldGetter::new(Self::type_index(), "thread_tag")?.get(self)
- }
+ }}
- pub fn span(&self) -> Result<Option<Span>> {
+ pub fn span(&self) -> Result<Option<Span>> {{
FieldGetter::new(Self::type_index(), "span")?.get(self)
- }
-}
+ }}
+}}
tvm_ffi::impl_object_upcast!(IterVar => PrimExprConvertible);"""
@@ -486,11 +557,11 @@ def test_render_iter_var_golden() -> None:
info = _info(
"tirx.IterVar",
(
- ("dom", TypeSchema("ir.Range")),
- ("var", TypeSchema("ir.Var")),
- ("iter_type", TypeSchema("int")),
- ("thread_tag", TypeSchema("str")),
- ("span", TypeSchema("ir.Span")),
+ _field("dom", "ir.Range"),
+ _field("var", "ir.Var"),
+ _field("iter_type", "int"),
+ _field("thread_tag", "str"),
+ _field("span", "ir.Span"),
),
parent="ir.PrimExprConvertible",
is_final=True,
@@ -518,7 +589,10 @@ def test_render_iter_var_golden() -> None:
def
test_field_directive_with_path_records_use_and_nullable_does_not_double_wrap()
-> None:
info = _info(
"demo.Node",
- (("buffer", TypeSchema("demo.Var")), ("dom", TypeSchema("Optional",
(TypeSchema("int"),)))),
+ (
+ _field("buffer", "demo.Var"),
+ _field("dom", TypeSchema("Optional", (TypeSchema("int"),))),
+ ),
)
imports = RUST.new_imports()
RUST.add_directive(imports, "field", "demo.Node.buffer ->
crate::typed::BufferVar", 1)
@@ -529,6 +603,376 @@ def
test_field_directive_with_path_records_use_and_nullable_does_not_double_wrap
assert "crate::typed::BufferVar" in _uses(imports)
+# ---------------------------------------------------------------------------
+# Complete rendering (byte facts prove the layout)
+# ---------------------------------------------------------------------------
+
+
+def _span() -> ObjectInfo:
+ return _info(
+ "ir.Span",
+ (
+ _field("source_name", "ir.SourceName", 24, 8),
+ _field("line", "int", 32, 4),
+ _field("column", "int", 36, 4),
+ _field("end_line", "int", 40, 4),
+ _field("end_column", "int", 44, 4),
+ ),
+ total_size=48,
+ )
+
+
+def _expr() -> ObjectInfo:
+ return _info(
+ "ir.Expr",
+ (_field("span", "ir.Span", 24, 8), _field("ty", "ir.Type", 32, 8)),
+ total_size=40,
+ )
+
+
+def _add() -> ObjectInfo:
+ return _info(
+ "tirx.Add",
+ (_field("a", "ir.Expr", 40, 8), _field("b", "ir.Expr", 48, 8)),
+ parent="ir.Expr",
+ total_size=56,
+ is_final=True,
+ )
+
+
+EXPR_EXPECTED = """\
+/// Complete: reflected fields fill [24, 40) exactly.
+#[repr(C)]
+#[derive(tvm_ffi::derive::Object)]
+#[type_key = "ir.Expr"]
+pub struct ExprObj {
+ base: Object,
+ pub span: Option<Span>,
+ pub ty: Type,
+}
+
+const _: () = {
+ assert!(::core::mem::size_of::<ExprObj>() == 40);
+ assert!(::core::mem::align_of::<ExprObj>() == 8);
+};
+
+#[repr(C)]
+#[derive(tvm_ffi::derive::ObjectRef, Clone)]
+pub struct Expr {
+ data: ObjectArc<ExprObj>,
+}
+
+impl Deref for Expr {
+ type Target = ExprObj;
+ fn deref(&self) -> &ExprObj {
+ &self.data
+ }
+}
+
+impl ExprObj {
+ pub(crate) fn new(span: Option<Span>, ty: Type) -> Self {
+ let base = Object::new();
+ Self { base, span, ty }
+ }
+}
+
+impl Expr {
+ /// Lossless complete-field allocation.
+ pub fn new(span: Option<Span>, ty: Type) -> Self {
+ let obj = ExprObj::new(span, ty);
+ Self { data: ObjectArc::new(obj) }
+ }
+}"""
+
+ADD_EXPECTED = """\
+/// Complete: reflected fields fill [40, 56) exactly.
+#[repr(C)]
+#[derive(tvm_ffi::derive::Object)]
+#[type_key = "tirx.Add"]
+#[type_final]
+pub struct AddObj {
+ base: ExprObj,
+ pub a: PrimExpr,
+ pub b: PrimExpr,
+}
+
+const _: () = {
+ assert!(::core::mem::size_of::<AddObj>() == 56);
+ assert!(::core::mem::align_of::<AddObj>() == 8);
+};
+
+#[repr(C)]
+#[derive(tvm_ffi::derive::ObjectRef, Clone)]
+pub struct Add {
+ data: ObjectArc<AddObj>,
+}
+
+impl Deref for Add {
+ type Target = AddObj;
+ fn deref(&self) -> &AddObj {
+ &self.data
+ }
+}
+
+impl Deref for AddObj {
+ type Target = ExprObj;
+ fn deref(&self) -> &ExprObj {
+ &self.base
+ }
+}
+
+impl AddObj {
+ pub(crate) fn new(span: Option<Span>, ty: PrimType, a: PrimExpr, b:
PrimExpr) -> Self {
+ let base = ExprObj::new(span, ty.into());
+ Self { base, a, b }
+ }
+}
+
+impl Add {
+ /// Lossless complete-field allocation.
+ pub fn new(span: Option<Span>, ty: PrimType, a: PrimExpr, b: PrimExpr) ->
Self {
+ let obj = AddObj::new(span, ty, a, b);
+ Self { data: ObjectArc::new(obj) }
+ }
+}
+
+tvm_ffi::impl_object_upcast!(Add => Expr, Add => PrimExpr);"""
+
+
+def test_render_complete_expr_golden() -> None:
+ """`ir.Expr` as tvm-rust-ext hand-writes it: `span: Option<Span>`, `ty:
Type`, no getters."""
+ imports = RUST.new_imports()
+ RUST.add_directive(imports, "nullable", "ir.Expr.span", 1)
+ text, imports = _render(_expr(), imports)
+ assert text == EXPR_EXPECTED
+ assert _uses(imports) == {"std::ops::Deref", "tvm_ffi::Object",
"tvm_ffi::ObjectArc"}
+
+
+def test_render_complete_add_golden() -> None:
+ """`tirx.Add` as tvm-rust-ext hand-writes it, on top of a complete
`ir.Expr`.
+
+ `field` directives narrow `a` / `b` and the inherited allocator parameter
+ `ty` (upcast with `.into()` on the way to `ExprObj::new`); `upcast` adds
+ the `PrimExpr` view.
+ """
+ _register(_expr())
+ imports = RUST.new_imports()
+ RUST.add_directive(imports, "nullable", "ir.Expr.span", 1)
+ RUST.add_directive(imports, "field", "tirx.Add.a -> PrimExpr", 2)
+ RUST.add_directive(imports, "field", "tirx.Add.b -> PrimExpr", 3)
+ RUST.add_directive(imports, "field", "tirx.Add.ty -> PrimType", 4)
+ RUST.add_directive(imports, "upcast", "tirx.Add -> PrimExpr", 5)
+ text, imports = _render(_add(), imports)
+ assert text == ADD_EXPECTED
+ assert _uses(imports) == {
+ "std::ops::Deref",
+ "tvm_ffi::ObjectArc",
+ "super::ir::ExprObj",
+ "super::ir::Expr",
+ "super::ir::Span",
+ "super::ir::Type",
+ }
+
+
+def test_render_complete_span_and_prim_type() -> None:
+ """Scalars take their reflected width; alignment padding is reported, not
mirrored."""
+ text, _ = _render(_span())
+ assert (
+ "pub struct SpanObj {\n"
+ " base: Object,\n"
+ " pub source_name: SourceName,\n"
+ " pub line: i32,\n"
+ " pub column: i32,\n"
+ " pub end_line: i32,\n"
+ " pub end_column: i32,\n"
+ "}"
+ ) in text
+ assert "/// Complete: reflected fields fill [24, 48) exactly." in text
+
+ _register(_info("ir.Type", (_field("span", "ir.Span", 24, 8),),
total_size=32))
+ prim_type = _info(
+ "ir.PrimType",
+ (_field("dtype", "dtype", 32, 4, 2),),
+ parent="ir.Type",
+ total_size=40,
+ is_final=True,
+ )
+ text, imports = _render(prim_type)
+ assert (
+ "/// Complete: reflected fields fill [32, 40) exactly (alignment
padding [36, 40))." in text
+ )
+ assert "pub struct PrimTypeObj {\n base: TypeObj,\n pub dtype:
DLDataType,\n}" in text
+ assert "assert!(::core::mem::size_of::<PrimTypeObj>() == 40);" in text
+ assert "tvm_ffi::DLDataType" in _uses(imports)
+
+
+def test_reserved_member_names_get_a_trailing_underscore() -> None:
+ """A reflected `base` or `data` field must not collide with the generated
members."""
+ info = _info(
+ "demo.Ramp",
+ (_field("base", "int", 24, 8), _field("data", "int", 32, 8)),
+ total_size=40,
+ is_final=True,
+ )
+ text, _ = _render(info)
+ assert " base: Object,\n pub base_: i64,\n pub data_: i64,\n" in
text
+ assert " pub fn new(base_: i64, data_: i64) -> Self {" in text
+ assert " Self { base, base_, data_ }" in text
+ # The opaque form keeps the reflected name on the C ABI side.
+ text, _ = _render(_info("demo.Node", (_field("base", "int"),)))
+ assert (
+ 'pub fn base_(&self) -> Result<i64> {\n
FieldGetter::new(Self::type_index(), "base")'
+ in text
+ )
+
+
+def test_render_complete_enum_field() -> None:
+ """An `enum` directive types the mirrored field; the newtype brings its
own `Result` import."""
+ info = _info(
+ "demo.Pair",
+ (_field("a", "int", 24, 8), _field("kind", "int", 32, 4)),
+ total_size=40,
+ is_final=True,
+ )
+ imports = RUST.new_imports()
+ RUST.add_directive(imports, "enum", "demo.Pair.kind -> Kind(i32) { A=0,
B=1 }", 1)
+ text, imports = _render(info, imports)
+ assert " pub kind: Kind,\n" in text
+ assert " pub fn new(a: i64, kind: Kind) -> Self {" in text
+ assert {"tvm_ffi::Result", "tvm_ffi::Error", "tvm_ffi::VALUE_ERROR"} <=
_uses(imports)
+ assert "tvm_ffi::FieldGetter" not in _uses(imports)
+
+
+def test_complete_optional_field_mirrors() -> None:
+ """`Optional<T>` fields mirror their C++ layout: a 16-byte cell or a
nullable pointer."""
+ info = _info(
+ "demo.Opt",
+ (
+ _field("count", TypeSchema("Optional", (TypeSchema("int"),)), 24,
16),
+ _field("name", TypeSchema("Optional", (TypeSchema("str"),)), 40,
16),
+ _field("items", TypeSchema("Optional", (TypeSchema("Array"),)),
56, 8),
+ ),
+ total_size=64,
+ )
+ text, imports = _render(info)
+ assert "pub count: Optional<i64>," in text
+ assert "pub name: Optional<String>," in text
+ assert "pub items: Option<Array<Any>>," in text
+ assert "tvm_ffi::Optional" in _uses(imports)
+
+
[email protected](
+ "field",
+ [
+ _field("x", TypeSchema("Optional", (TypeSchema("Any"),)), 24, 16),
+ _field("x", TypeSchema("Optional", (TypeSchema("int"),)), 24, 8), #
not the 16-byte cell
+ _field("x", TypeSchema("Union", (TypeSchema("int"),
TypeSchema("str"))), 24, 16),
+ _field("x", "ctypes.c_void_p", 24, 8),
+ ],
+)
+def test_unrenderable_field_keeps_the_type_opaque(field: NamedTypeSchema) ->
None:
+ """A field without a mirror demotes an otherwise complete type; it is read
as `Any`."""
+ assert field.size is not None
+ info = _info("demo.Holder", (field,), total_size=HEADER + field.size)
+ text, _ = _render(info)
+ assert "/// Opaque: field 'x'" in text
+ assert "has no native mirror" in text
+ assert "impl HolderObj {\n pub fn x(&self) -> Result<" in text
+ assert "const _: () =" not in text
+
+
+def test_custom_new_renames_the_wrapper_allocator() -> None:
+ """`custom-new`: `Add::new` stays hand-written, the allocator is
`from_complete_fields`."""
+ _register(_expr())
+ imports = RUST.new_imports()
+ RUST.add_directive(imports, "custom-new", "tirx.Add", 1)
+ text, _ = _render(_add(), imports)
+ assert (
+ "impl AddObj {\n pub(crate) fn new(span: Span, ty: Type, a: Expr,
b: Expr) -> Self {"
+ in text
+ )
+ assert (
+ "impl Add {\n /// Lossless complete-field allocation.\n"
+ " pub fn from_complete_fields(span: Span, ty: Type, a: Expr, b:
Expr) -> Self {\n"
+ " let obj = AddObj::new(span, ty, a, b);\n"
+ " Self { data: ObjectArc::new(obj) }\n"
+ " }\n}"
+ ) in text
+ assert " pub fn new(" not in text
+
+
+def test_upcast_directive_adds_typed_views() -> None:
+ """`upcast` targets follow the ancestor chain; a `::` path is imported.
Opaque: no allocator."""
+ info = _info("demo.Leaf", parent="demo.Base")
+ imports = RUST.new_imports()
+ RUST.add_directive(imports, "upcast", "demo.Leaf ->
crate::typed::LeafView", 1)
+ RUST.add_directive(imports, "upcast", "demo.Leaf -> Other", 2)
+ text, imports = _render(info, imports)
+ assert text.endswith(
+ "tvm_ffi::impl_object_upcast!(Leaf => Base, Leaf => LeafView, Leaf =>
Other);"
+ )
+ assert "crate::typed::LeafView" in _uses(imports)
+ assert "fn new(" not in text
+
+
+def test_opaque_directive_vetoes_a_complete_type() -> None:
+ imports = RUST.new_imports()
+ RUST.add_directive(imports, "opaque", "ir.Span", 1)
+ text, _ = _render(_span(), imports)
+ assert "/// Opaque: vetoed by directive although the layout is
reproducible." in text
+ assert "pub fn line(&self) -> Result<i64> {" in text
+
+
+def test_builtin_parent_keeps_the_type_opaque() -> None:
+ """The crate never mirrors a builtin's bytes: the header-only stand-in
cannot be complete."""
+ # Fieldless under `ffi.Enum` (48 bytes): the fill criterion alone would
call this complete.
+ info = _info(
+ "demo.Flag", parent="ffi.Enum", ancestors=["ffi.Object", "ffi.Enum"],
total_size=48
+ )
+ text, _ = _render(info)
+ assert "/// Opaque: parent 'ffi.Enum' is opaque (no-mirror)." in text
+ assert " base: FfiEnumObj," in text
+ assert "const _: () =" not in text
+ assert "fn new(" not in text
+ # The registry fixtures under `ffi.Enum` / `ffi.IntEnum` / `ffi.StrEnum`
follow the same rule.
+ for type_key, parent in (
+ ("testing.TestEnumVariant", "ffi.Enum"),
+ ("testing.TestCxxIntEnum", "ffi.IntEnum"),
+ ("testing.TestCxxStrEnum", "ffi.StrEnum"),
+ ):
+ text, _ = _render(object_info_from_type_key(type_key))
+ assert f"/// Opaque: parent '{parent}' is opaque (no-mirror)." in text
+ assert "const _: () =" not in text
+ assert "fn new(" not in text
+
+
+def test_opaque_parent_keeps_the_child_opaque() -> None:
+ """A child of an opaque type cannot embed a mirror of its parent: it stays
opaque."""
+ _register(_info("ir.Expr", (_field("span", "ir.Span", 24, 8),),
total_size=40)) # hole
+ text, _ = _render(_add())
+ assert "/// Opaque: parent 'ir.Expr' is opaque (uncovered-bytes)." in text
+ assert " base: ExprObj," in text
+ assert "pub fn a(&self) -> Result<Expr> {" in text
+
+
[email protected](
+ ("name", "payload", "message"),
+ [
+ ("field", "demo.Pair.count -> i64", "maps a 4-byte field to `i64` (8
bytes)"),
+ ("enum", "demo.Pair.count -> Kind(i64)", "maps a 4-byte field to `i64`
(8 bytes)"),
+ ("nullable", "demo.Pair.count", "the field is 4 bytes, not a
pointer-sized"),
+ ],
+)
+def test_directive_disagreeing_with_bytes_is_an_error(
+ name: str, payload: str, message: str
+) -> None:
+ info = _info("demo.Pair", (_field("count", "int", 24, 4),), total_size=32)
+ imports = RUST.new_imports()
+ RUST.add_directive(imports, name, payload, 1)
+ with pytest.raises(ValueError, match=re.escape(message)):
+ _render(info, imports)
+
+
# ---------------------------------------------------------------------------
# File scaffolding
# ---------------------------------------------------------------------------
@@ -578,10 +1022,61 @@ def test_finalize_module_tree(tmp_path: Path) -> None:
# ---------------------------------------------------------------------------
-# The pipeline end to end
+# The real registry, and the pipeline end to end
# ---------------------------------------------------------------------------
+def test_registry_complete_chain_is_mirrored() -> None:
+ base, _ = _render(object_info_from_type_key("testing.TestCxxClassBase"))
+ assert (
+ "/// Complete: reflected fields fill [24, 40) exactly (alignment
padding [36, 40))." in base
+ )
+ assert (
+ "pub struct TestCxxClassBaseObj {\n base: Object,\n pub v_i64:
i64,\n pub v_i32: i32,\n}"
+ in base
+ )
+ assert "assert!(::core::mem::size_of::<TestCxxClassBaseObj>() == 40);" in
base
+ assert "FieldGetter" not in base
+ assert " pub fn new(v_i64: i64, v_i32: i32) -> Self {" in base
+
+ dd, imports =
_render(object_info_from_type_key("testing.TestCxxClassDerivedDerived"))
+ assert (
+ " base: TestCxxClassDerivedObj,\n pub v_str: String,\n pub
v_bool: bool,\n}" in dd
+ )
+ # The allocator flattens the chain; a signature over 100 columns wraps.
+ assert (
+ "impl TestCxxClassDerivedDerivedObj {\n"
+ " pub(crate) fn new(\n"
+ " v_i64: i64,\n"
+ " v_i32: i32,\n"
+ " v_f64: f64,\n"
+ " v_f32: f32,\n"
+ " v_str: String,\n"
+ " v_bool: bool,\n"
+ " ) -> Self {\n"
+ " let base = TestCxxClassDerivedObj::new(v_i64, v_i32, v_f64,
v_f32);\n"
+ " Self { base, v_str, v_bool }\n"
+ " }\n"
+ "}"
+ ) in dd
+ assert dd.endswith(
+ "tvm_ffi::impl_object_upcast!(TestCxxClassDerivedDerived =>
TestCxxClassBase, "
+ "TestCxxClassDerivedDerived => TestCxxClassDerived);"
+ )
+ assert "tvm_ffi::String" in _uses(imports)
+
+
+def test_registry_hidden_field_and_vptr_stay_opaque() -> None:
+ hidden, _ =
_render(object_info_from_type_key("testing.TestCxxClassHiddenField"))
+ assert (
+ "/// Opaque: bytes [32, 40) of [24, 48) are not accounted for by
reflected fields. "
+ "Fields are read through the C ABI getters."
+ ) in hidden
+ assert "pub fn v_i32(&self) -> Result<i64> {" in hidden
+ poly, _ =
_render(object_info_from_type_key("testing.TestCxxClassPolymorphic"))
+ assert "/// Opaque: bytes [32, 40) of [24, 40)" in poly
+
+
def test_stage_3_applies_directives_to_a_registered_type(tmp_path: Path) ->
None:
src = tmp_path / "mod.rs"
src.write_text(
@@ -602,9 +1097,8 @@ def
test_stage_3_applies_directives_to_a_registered_type(tmp_path: Path) -> None
_stage_3(info, Options(dry_run=True), RUST.default_ty_map(), {}, RUST)
text = "\n".join(line for block in info.code_blocks for line in
block.lines)
assert "pub struct Kind(i32);" in text
- assert "pub fn v_i32(&self) -> Result<Kind> {" in text
- assert "pub fn v_i64(&self) -> Result<i64> {" in text
- assert "use tvm_ffi::FieldGetter;" in text
+ assert " pub v_i32: Kind,\n" in text
+ assert "use tvm_ffi::Error;" in text
def test_cli_init_generates_a_module_tree(tmp_path: Path, monkeypatch:
pytest.MonkeyPatch) -> None:
@@ -627,9 +1121,9 @@ def test_cli_init_generates_a_module_tree(tmp_path: Path,
monkeypatch: pytest.Mo
assert (tmp_path / "mod.rs").read_text(encoding="utf-8") == "pub mod
testing;\n"
text = (tmp_path / "testing" / "mod.rs").read_text(encoding="utf-8")
assert text.startswith("#![allow(dead_code, unused_imports)]\n")
- assert "use tvm_ffi::FieldGetter;" in text
+ assert "use tvm_ffi::FieldGetter;" in text # the opaque fixtures read
through getters
assert '#[type_key = "testing.TestCxxClassDerivedDerived"]' in text
- assert " base: TestCxxClassDerivedObj," in text
+ assert " base: TestCxxClassDerivedObj,\n pub v_str: String," in text
assert (
"tvm_ffi::impl_object_upcast!(TestCxxClassDerivedDerived =>
TestCxxClassBase, "
"TestCxxClassDerivedDerived => TestCxxClassDerived);"
