tqchen commented on PR #744: URL: https://github.com/apache/tvm-ffi/pull/744#issuecomment-5548773864
Correction to the measurements above, from re-running the A/B against the exact pushed head. The **−6.9% on the replace arm does not reproduce.** On the final branch state it measures +0.6%, and its baseline also shifted from 22.79 to 21.23 ns/node between runs — same `origin/main` library, different benchmark binary. Do not rely on that row. What holds, 13 interleaved pairs against identically built libraries: | arm | base | new | delta | |---|---:|---:|---:| | minimal-vtable floor *(control)* | 8.84 | 8.98 | +0.14 (+1.6%) | | no link matches | 14.56 | **13.88** | **−0.68 (−4.7%)** | | link matches, no rebuild | 21.83 | **20.91** | **−0.92 (−4.2%)** | | link matches, node replaced | 21.23 | 21.36 | +0.13 (+0.6%) | The floor is a control that should not move, and its +1.6% drift sets the real noise level at about ±0.15 ns/node. The two improvements are 4–6× that and their medians are well separated; the replace row is inside it. Two caveats on this harness, both found the hard way and both worth knowing before trusting any number from it: - Absolute values shift with code layout. Registering unrelated hooks in the same binary moved a number by 17%. - Arms timed sequentially in one process are not comparable either. In the current binary the replace arm measures *cheaper* than the no-rebuild arm, which is backwards, and the difference tracks position in `main()` rather than work done. The codegen effect is not subject to either problem and is unambiguous: per-node dispatch frame 216 → 120 bytes and 358 → 318 instructions, maybe-in-place 232 → 120 and 421 → 352, with both error builders emitted into `.text.unlikely`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
