Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 merged PR #254: URL: https://github.com/apache/tvm-ffi/pull/254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
Kathryn-cat commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-4137258466 thanks @cyx-6 ! the PR is LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
gemini-code-assist[bot] commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2996868697
##
addons/tvm-ffi-orcjit/README.md:
##
@@ -0,0 +1,261 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+# TVM-FFI OrcJIT
+
+A Python package that enables dynamic loading of compiled object files (`.o`)
+using LLVM ORC JIT v2, providing a flexible JIT execution environment for
+TVM-FFI exported functions.
+
+## Features
+
+- **JIT Execution**: Load and execute compiled object files at runtime using
LLVM's ORC JIT v2
+- **Multiple Libraries**: Create separate dynamic libraries with independent
symbol namespaces
+- **Incremental Loading**: Add multiple object files to the same library
incrementally
+- **Symbol Isolation**: Different libraries can define the same symbol without
conflicts
+- **Init/Fini Support**: Handles static constructors/destructors across ELF
(`.init_array`/`.ctors`), Mach-O (`__mod_init_func`), and COFF
(`.CRT$XC*`/`.CRT$XT*`)
+- **Cross-Platform**: Linux (x86_64, aarch64), macOS (arm64), Windows (AMD64)
+- **Multi-Compiler**: Tested with LLVM Clang, GCC, Apple Clang, MSVC, and
clang-cl
+- **TVM-FFI Integration**: Seamlessly works with TVM-FFI's stable C ABI
+- **Python API**: Simple Pythonic interface for JIT compilation and execution
+
+## Supported Platforms and Compilers
+
+Object files compiled with any of the following compiler/platform combinations
+can be loaded and executed by the ORC JIT:
+
+| Platform | Compilers | C | C++ |
+| | - | :-: | :-: |
+| Linux (x86_64, aarch64) | LLVM Clang, GCC | yes | yes |
+| macOS (arm64) | LLVM Clang, Apple Clang | yes | yes |
+| Windows (AMD64) | LLVM Clang, MSVC, clang-cl | yes | no |
+
+Windows is C-only across all compilers. C++ objects compiled with
+`TVM_FFI_DLL_EXPORT_TYPED_FUNC` use `try`/`catch` (via
`TVM_FFI_SAFE_CALL_BEGIN/END`),
+which requires Itanium exception ABI symbols (`__cxa_begin_catch`,
+`__gxx_personality_v0`, etc.) that the MSVC-built host process cannot provide.
+Pure C objects using the `TVMFFISafeCallType` ABI work on all platforms.
+
+## Installation
+
+### Install from PyPI
+
+```bash
+pip install apache-tvm-ffi apache-tvm-ffi-orcjit
+```
+
+### Build from Source
+
+ Prerequisites
+
+- Python 3.10+, CMake 3.20+, C++17 compiler
+- LLVM 22+ development libraries (`llvmdev`, `llvm-config`)
+- Static `zlib` and `zstd` libraries (in the same prefix as LLVM)
+
+ Install LLVM via conda-forge
+
+The easiest way to get all dependencies is via conda-forge:
+
+```bash
+conda create -p /opt/llvm -c conda-forge \
+ llvmdev=22.1.0 clangdev=22.1.0 compiler-rt=22.1.0 zlib zstd-static -y
+export LLVM_PREFIX=/opt/llvm
+```
+
+On Windows:
+
+```cmd
+conda create -p C:\opt\llvm -c conda-forge llvmdev=22.1.0 zlib zstd-static -y
+set LLVM_PREFIX=C:\opt\llvm
+```
+
+ Build and install
+
+```bash
+git clone --recursive https://github.com/apache/tvm-ffi.git
+cd tvm-ffi
+
+# Install tvm-ffi first
+pip install -e .
+
+# Build and install the orcjit addon
+cd addons/tvm-ffi-orcjit
+pip install -e .
+```
+
+The `LLVM_PREFIX` environment variable tells CMake where to find LLVM. If
+LLVM is installed in a conda env or a standard system path, CMake can
+auto-discover it and `LLVM_PREFIX` is not needed.
+
+## Usage
+
+### Basic Example
+
+```python
+from tvm_ffi_orcjit import ExecutionSession
+
+# Create an execution session
+session = ExecutionSession()
+
+# Create a dynamic library
+lib = session.create_library()
+
+# Load an object file
+lib.add("example.o")
+
+# Get and call a function
+add_func = lib.get_function("add")
+result = add_func(1, 2)
+print(f"Result: {result}") # Output: Result: 3
+```
+
+### Multiple Libraries with Symbol Isolation
+
+```python
+session = ExecutionSession()
+
+lib1 = session.create_library("lib1")
+lib2 = session.create_library("lib2")
+
+lib1.add("implementation_v1.o")
+lib2.add("implementation_v2.o")
+
+add_v1 = lib1.get_function("add")
+add_v2 = lib2.get_function("add")
+
+print(add_v1(5, 3)) # Uses implementation from lib1
+print(add_v2(5, 3)) # Uses implementation from lib2
+```
+
+### Cross-Library Linking
+
+```python
+session = ExecutionSession()
+
+base_lib = session.create_library("base")
+base_lib.add("math_ops.o")
+
+caller_lib = session.create_library("caller")
+caller_lib.set_link_order(base_lib) # Can resolve symbols from base_lib
+caller_lib.add("caller.o")
+
+result = caller_lib.get_function("call_math")(10, 20)
+```
+
+## Writing Functions for OrcJIT
+
+### C++ (Linux/macOS)
+
+```cpp
+#include
+
+TVM_FFI_DLL_EXPORT_TYPED_FUNC(add, [](int a, int b) {
+return a + b;
+});
+```
+
+Compile: `clang++ -std=c++17 -fPIC -O2 -c -o example.o example.cc`
+
+### Pure C (all platforms including Windows)
+
+```c
+#include
+
+TVM_FFI_DLL_EXPORT int __tvm_ffi_add(
+void* self, const TVMFFIAny* args, int32_t num_args, TVMFFIAny* result) {
+ result->type_index = kTVMFFIInt;
+ result->v_int64 = args[0].v_int64 + args[1].v_int64;
+ re
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-4137226387 /gemini review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2996842669
##
addons/tvm-ffi-orcjit/tests/sources/cuda/test_funcs.cu:
##
@@ -0,0 +1,61 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+
+#include
+
+// Simple addition function
+__global__ void test_add_kernel(int* a, int* b, int* c) { *c = *a + *b; }
+int test_add_impl(int a, int b) {
+ int c;
+ int *d_a, *d_b, *d_c;
+ cudaMalloc(&d_a, sizeof(int));
+ cudaMalloc(&d_b, sizeof(int));
Review Comment:
a betetr canonical examplw would be apssing DLTensor in from python side via
pytorch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yaoyaoding commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-4136034641 The PR looks good to me now, thanks @cyx-6 ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yaoyaoding commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2995862445
##
addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/session.py:
##
@@ -0,0 +1,90 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""ORC JIT Execution Session."""
+
+from __future__ import annotations
+
+import sys
+
+from tvm_ffi import Object, register_object
+
+from . import _ffi_api, _lib_dir
+from .dylib import DynamicLibrary
+
+
+def _find_orc_rt_library() -> str | None:
+"""Find the bundled liborc_rt library in the same directory as the
.so/.dll."""
+# Windows: skip ORC runtime entirely. LLVM's COFFPlatform (loaded via
+# ExecutorNativePlatform with liborc_rt) depends on MSVC C++ runtime
symbols
+# that are not available in the JIT environment. On Windows, ORC JIT uses a
+# C-only strategy: JIT objects are compiled as pure C (TVMFFISafeCallType
ABI),
+# avoiding all C++ runtime dependencies (magic statics, RTTI, sized delete,
+# SEH, COMDAT). Our custom InitFiniPlugin handles .CRT$XC*/.CRT$XT*
init/fini
+# sections, and DLLImportDefinitionGenerator resolves __imp_ DLL import
stubs.
+if sys.platform == "win32":
+return None
+patterns = ["liborc_rt*.a"]
+for pattern in patterns:
+for lib_path in _lib_dir.glob(pattern):
+return str(lib_path)
+return None
+
+
+@register_object("orcjit.ExecutionSession")
+class ExecutionSession(Object):
+"""ORC JIT Execution Session.
+
+Manages the LLVM ORC JIT execution environment and creates dynamic
libraries (JITDylibs).
+This is the top-level context for JIT compilation and symbol management.
+
+Examples
+
+>>> session = ExecutionSession()
+>>> lib = session.create_library(name="main")
+>>> lib.add("add.o")
+>>> add_func = lib.get_function("add")
+
+"""
+
+def __init__(self, orc_rt_path: str | None = None) -> None:
+"""Initialize ExecutionSession.
+
+Args:
+orc_rt_path: Optional path to the liborc_rt library. If not
provided,
+it will be automatically discovered using clang.
+
+"""
+if orc_rt_path is None:
+orc_rt_path = _find_orc_rt_library()
+if orc_rt_path is None:
+orc_rt_path = ""
+self.__init_handle_by_constructor__(_ffi_api.ExecutionSession,
orc_rt_path) # type: ignore
+
+def create_library(self, name: str = "") -> DynamicLibrary:
+"""Create a new dynamic library associated with this execution session.
+
+Args:
+name: Optional name for the library. If empty, a unique name will
be generated.
+
+Returns:
+A new DynamicLibrary instance.
+
+"""
+handle = _ffi_api.ExecutionSessionCreateDynamicLibrary(self, name) #
type: ignore
+lib = DynamicLibrary.__new__(DynamicLibrary)
+lib.__move_handle_from__(handle)
Review Comment:
I see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yaoyaoding commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-4135923536 > The problem is that COFFPlatform requires a COFF ORC runtime library (orc_rt) to be built and loaded. > ... > In principle, if someone completes the COFF ORC runtime (especially the exception interop), COFFPlatform could work. But that's been stalled for 2+ years in LLVM. Our current approach (`InitFiniPlugin` + `DLLImportDefinitionGenerator`) gives us working C support without depending on the incomplete runtime. Thank you for the clarification! Looks like we are on the edge of the orcjit's current development & support coverage. It's pretty cool you finally make it work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2995327438 ## addons/tvm-ffi-orcjit/tests/run_all_tests.py: ## @@ -0,0 +1,81 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Build test objects, run pytest, and run quick-start examples. + +Single entry point for CI. Replaces the separate cmake build step and +platform-specific test commands. + +Usage: +python run_all_tests.py [--llvm-prefix /opt/llvm] +""" + +from __future__ import annotations + +import argparse Review Comment: tvm_ffi.cpp.build is now refactored, matching the torch api and supporting the object files as input -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2995329414
##
addons/tvm-ffi-orcjit/tests/build_test_objects.py:
##
@@ -0,0 +1,319 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Build test objects and quick-start example via direct compiler invocation.
+
+Detects platform and available compilers, then builds all applicable variants:
+ Linux: LLVM Clang (default) + GCC
+ macOS: LLVM Clang (default) + Apple Clang
+ Windows: LLVM Clang (default) + MSVC + clang-cl
+
+Objects are cached: files are only recompiled when the source is newer than the
+output. Re-running this script is near-instant when nothing changed.
+
+Usage:
+python build_test_objects.py [--llvm-prefix /opt/llvm]
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import platform
+import shutil
+import subprocess
+import sys
+from pathlib import Path
+
+TESTS_DIR = Path(__file__).resolve().parent
+ADDON_DIR = TESTS_DIR.parent
+QUICKSTART_DIR = ADDON_DIR / "examples" / "quick-start"
+SOURCES_C = TESTS_DIR / "sources" / "c"
+SOURCES_CC = TESTS_DIR / "sources" / "cc"
+
+
+# ---
+# Helpers
+# ---
+
+
+def _get_tvm_ffi_includedir() -> str:
+"""Return the tvm-ffi C/C++ include directory."""
+result = subprocess.run(
+[sys.executable, "-m", "tvm_ffi.config", "--includedir"],
+capture_output=True,
+text=True,
+check=True,
+)
+return result.stdout.strip()
+
+
+def _needs_build(source: Path, output: Path) -> bool:
+"""Check whether *output* is missing or older than *source*."""
+if not output.exists():
+return True
+return source.stat().st_mtime > output.stat().st_mtime
+
+
+def _compile(
+compiler: str, source: Path, output: Path, flags: list[str], include_dirs:
list[str]
+) -> None:
+"""Compile *source* → *output*. Skips if cached."""
+if not _needs_build(source, output):
+return
+output.parent.mkdir(parents=True, exist_ok=True)
+cmd: list[str] = [compiler, *flags]
+for d in include_dirs:
+if compiler in ("cl", "cl.exe", "clang-cl", "clang-cl.exe"):
+cmd.append(f"/I{d}")
+else:
+cmd += ["-I", d]
+if compiler in ("cl", "cl.exe", "clang-cl", "clang-cl.exe"):
+cmd += ["/c", f"/Fo{output}", str(source)]
+else:
+cmd += ["-c", "-o", str(output), str(source)]
+print(f" {' '.join(cmd)}", flush=True)
+subprocess.check_call(cmd)
+
+
+# ---
+# Variant builder
+# ---
+
+
+def _build_variant(
+name: str,
+*,
+c_compiler: str,
+cxx_compiler: str | None,
+c_flags: list[str],
+cxx_flags: list[str],
+include_dirs: list[str],
+c_outdir: Path,
+cc_outdir: Path | None,
+) -> None:
+"""Build all test objects for one compiler variant."""
+print(f"\n--- {name} ---", flush=True)
+for src in sorted(SOURCES_C.glob("*.c")):
+_compile(c_compiler, src, c_outdir / f"{src.stem}.o", c_flags,
include_dirs)
+if cxx_compiler and cc_outdir:
+for src in sorted(SOURCES_CC.glob("*.cc")):
+_compile(cxx_compiler, src, cc_outdir / f"{src.stem}.o",
cxx_flags, include_dirs)
+
+
+# ---
+# Platform dispatch
+# ---
+
+
+def _default_llvm_prefix() -> str:
+if platform.system() == "Windows":
+return "C:/opt/llvm"
+return "/opt/llvm"
+
+
+def _find_llvm_bin(prefix: str) -> Path:
+"""Return the LLVM bin directory under *prefix*."""
+p = Path(prefix)
+# conda-forge on Windows: Library/bin; elsewhere: bin
+win_lib = p / "Library" / "bin"
+if win_lib.exists():
+return win_lib
+return p / "bin"
+
+
+def _base_flags(system: str, machine: str) -> tuple[list[str], list[str]]:
+"""Return (c_flags, cxx_flags) base
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2995321245
##
addons/tvm-ffi-orcjit/scripts/install_llvm.ps1:
##
@@ -0,0 +1,98 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Install LLVM from conda-forge using micromamba (Windows).
+# Usage: powershell -ExecutionPolicy Bypass -File tools/install_llvm.ps1
[version]
+# version defaults to LLVM_VERSION env var, then 22.1.0
+
+param(
+[string]$Version = ""
+)
+
+$ErrorActionPreference = "Stop"
+Set-StrictMode -Version Latest
+
+if (-not $Version) {
+$Version = if ($env:LLVM_VERSION) { $env:LLVM_VERSION } else { "22.1.0" }
+}
+$Prefix = if ($env:LLVM_PREFIX) { $env:LLVM_PREFIX } else { "C:\opt\llvm" }
+
+Write-Host "Installing LLVM $Version to $Prefix"
+
+# Install micromamba from GitHub releases (micro.mamba.pm cert expired as of
2026-03)
+$MicromambaExe = "$env:TEMP\micromamba.exe"
+
+if (-not (Test-Path $MicromambaExe)) {
+Write-Host "Downloading micromamba from GitHub releases..."
+$maxRetries = 3
+for ($attempt = 1; $attempt -le $maxRetries; $attempt++) {
+try {
+& curl.exe -sSL -o $MicromambaExe
"https://github.com/mamba-org/micromamba-releases/releases/latest/download/micromamba-win-64.exe";
+if ($LASTEXITCODE -ne 0) { throw "curl failed with exit code
$LASTEXITCODE" }
+break
+} catch {
+Write-Host "Attempt $attempt/$maxRetries failed: $_"
+if ($attempt -eq $maxRetries) { throw }
+Start-Sleep -Seconds 5
+}
+}
+}
+Write-Host "Using micromamba: $MicromambaExe"
+
+# Install LLVM and zlib. No clangdev or compiler-rt on Windows — test objects
+# use C-only strategy compiled with the system compiler (MSVC), and liborc_rt
+# is not used (Windows ORC JIT skips COFFPlatform).
+& $MicromambaExe create -p $Prefix -c conda-forge `
+"llvmdev=$Version" `
+zlib `
+-y
+if ($LASTEXITCODE -ne 0) { throw "micromamba create failed" }
+
+# Build static zstd from source.
Review Comment:
scripts removed in new workflow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2995318666 ## addons/tvm-ffi-orcjit/scripts/install_llvm.sh: ## @@ -0,0 +1,73 @@ +#!/bin/bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Install LLVM from conda-forge using micromamba. +# Usage: bash tools/install_llvm.sh [version] +# version defaults to LLVM_VERSION env var, then 22.1.0 +set -ex Review Comment: simplified -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2993283996
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+ *ctx_addr = symbol;
+}
+ });
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+ORCJITDynamicLibraryObj::~ORCJITDynamicLibraryObj() {
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run section-based deinitializers (.fini_array, .dtors,
.CRT$XT*)
+ // collected by our custom InitFiniPlugin.
+ session_->RunPendingDeinitializers(GetJITDylib());
+#else
+ // macOS: native platform's deinitialize drains __cxa_atexit handlers
+ // (registered during initialization) via the ORC runtime.
+ if (auto err = jit_->deinitialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+}
+
+void ORCJITDynamicLibraryObj::AddObjectFile(const String& path) {
+ // Read object file
+ auto buffer_or_err = llvm::MemoryBuffer::getFile(path.c_str());
+ if (!buffer_or_err) {
+TVM_FFI_THROW(IOError) << "Failed to read object file: " << path;
+ }
+
+ // Add object file to this JITDylib
+ call_llvm(jit_->addObjectFile(*dylib_, std::move(*buffer_or_err)), "Failed
to add object file");
+}
+
+void ORCJITDynamicLibraryObj::SetLinkOrder(const
std::vector& dylibs) {
+ // Rebuild the link order: user-specified libraries first, then the LLJIT
+ // default link order (Main → Platform → ProcessSymbols). Preserving the
+ // default link order is essential — without ProcessSymbols, C++ objects
+ // that need host-process symbols (runtime, libtvm_ffi) would fail to link.
+ link_order_.clear();
+
+ for (auto* lib : dylibs) {
+link_order_.emplace_back(lib,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ }
+ for (auto& kv : jit_->defaultLinkOrder()) {
+link_order_.emplace_back(kv.first, kv.second);
+ }
+
+ // Set the link order in the LLVM JITDylib
+ dylib_->setLinkOrder(link_order_, false);
+}
+
+void* ORCJITDynamicLibraryObj::GetSymbol(const String& name) {
+ // Build search order: this dylib first, then all linked dylibs
+ llvm::orc::JITDylibSearchOrder search_order;
+ search_order.emplace_back(dylib_,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ // Append linked libraries
+ search_order.insert(search_order.end(), link_order_.begin(),
link_order_.end());
+
+ // Look up symbol using the full search order
+ auto symbol_or_err =
+ jit_->getExecutionSession().lookup(search_order,
jit_->mangleAndIntern(name.c_str()));
+
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run initializers collected by our custom InitFiniPlugin.
+ session_->RunPendingInitializers(GetJITDylib());
+#else
+ // macOS: use native platform's init mechanism (handles __mod_init_func
+ // and __cxa_atexit registration).
+ if (auto err = jit_->initialize(*dylib_)) {
Review Comment:
Looked into this more carefully — the refcount actually stays balanced.
LLJIT's `ORCPlatformSupport::initialize()` tracks an `InitializedDylib` set.
The first call invokes `dlopen`
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-4132649650 > If we load the MSVC CRT into the process, will the COFFPlatform work? The MSVC CRT DLLs are already loaded in the host process — our DLLImportDefinitionGenerator resolves symbols from vcruntime140, ucrtbase, msvcp140, etc. The problem is that COFFPlatform requires a COFF ORC runtime library (orc_rt) to be built and loaded. Looking at the LLVM source, COFFPlatform needs to resolve ~10 runtime entry points like `__orc_rt_coff_platform_bootstrap`, `__orc_rt_coff_register_jitdylib`, `__orc_rt_coff_register_object_sections`, etc. It also aliases `_CxxThrowException` -> `__orc_rt_coff_cxx_throw_exception` The COFF ORC runtime does exist in compiler-rt, and it implements `__orc_rt_coff_cxx_throw_exception` — but with a big caveat: cross-module exception throwing is unfinished. So even if we build and load the COFF orc_rt, exception handling between host code and JIT code would abort(). The LLVM test suite does include COFF ORC tests (e.g., sehframe-handling.cpp, hello-world.cpp in compiler-rt/test/orc/TestCases/Windows/x86-64/), but these require a fully functional COFF runtime which hasn't been completed. In principle, if someone completes the COFF ORC runtime (especially the exception interop), COFFPlatform could work. But that's been stalled for 2+ years in LLVM. Our current approach (`InitFiniPlugin` + `DLLImportDefinitionGenerator`) gives us working C support without depending on the incomplete runtime. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yaoyaoding commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-4128070854 > Would be return to ELFNixPlatform once new llvm release contains this patch commit. Cool! I saw your PR to the llvm. > requires MSVC CRT symbols If we load the MSVC CRT into the process, will the COFFPlatform work? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yaoyaoding commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2989549770
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+ *ctx_addr = symbol;
+}
+ });
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+ORCJITDynamicLibraryObj::~ORCJITDynamicLibraryObj() {
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run section-based deinitializers (.fini_array, .dtors,
.CRT$XT*)
+ // collected by our custom InitFiniPlugin.
+ session_->RunPendingDeinitializers(GetJITDylib());
+#else
+ // macOS: native platform's deinitialize drains __cxa_atexit handlers
+ // (registered during initialization) via the ORC runtime.
+ if (auto err = jit_->deinitialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+}
+
+void ORCJITDynamicLibraryObj::AddObjectFile(const String& path) {
+ // Read object file
+ auto buffer_or_err = llvm::MemoryBuffer::getFile(path.c_str());
+ if (!buffer_or_err) {
+TVM_FFI_THROW(IOError) << "Failed to read object file: " << path;
+ }
+
+ // Add object file to this JITDylib
+ call_llvm(jit_->addObjectFile(*dylib_, std::move(*buffer_or_err)), "Failed
to add object file");
+}
+
+void ORCJITDynamicLibraryObj::SetLinkOrder(const
std::vector& dylibs) {
+ // Rebuild the link order: user-specified libraries first, then the LLJIT
+ // default link order (Main → Platform → ProcessSymbols). Preserving the
+ // default link order is essential — without ProcessSymbols, C++ objects
+ // that need host-process symbols (runtime, libtvm_ffi) would fail to link.
+ link_order_.clear();
+
+ for (auto* lib : dylibs) {
+link_order_.emplace_back(lib,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ }
+ for (auto& kv : jit_->defaultLinkOrder()) {
+link_order_.emplace_back(kv.first, kv.second);
+ }
+
+ // Set the link order in the LLVM JITDylib
+ dylib_->setLinkOrder(link_order_, false);
+}
+
+void* ORCJITDynamicLibraryObj::GetSymbol(const String& name) {
+ // Build search order: this dylib first, then all linked dylibs
+ llvm::orc::JITDylibSearchOrder search_order;
+ search_order.emplace_back(dylib_,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ // Append linked libraries
+ search_order.insert(search_order.end(), link_order_.begin(),
link_order_.end());
+
+ // Look up symbol using the full search order
+ auto symbol_or_err =
+ jit_->getExecutionSession().lookup(search_order,
jit_->mangleAndIntern(name.c_str()));
+
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run initializers collected by our custom InitFiniPlugin.
+ session_->RunPendingInitializers(GetJITDylib());
+#else
+ // macOS: use native platform's init mechanism (handles __mod_init_func
+ // and __cxa_atexit registration).
+ if (auto err = jit_->initialize(*dylib_)) {
Review Comment:
Will this make the ref count always > 0? like we initialize it many times
and only deinitialize once when unloading?
--
This is an automated message from the Apache Git Ser
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986948638
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+ *ctx_addr = symbol;
+}
+ });
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+ORCJITDynamicLibraryObj::~ORCJITDynamicLibraryObj() {
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run section-based deinitializers (.fini_array, .dtors,
.CRT$XT*)
+ // collected by our custom InitFiniPlugin.
+ session_->RunPendingDeinitializers(GetJITDylib());
+#else
Review Comment:
doc updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986922999
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_session.cc:
##
@@ -0,0 +1,664 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_session.cc
+ * \brief LLVM ORC JIT ExecutionSession implementation
+ */
+
+#include "orcjit_session.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+
+#ifdef _WIN32
+#ifndef NOMINMAX
+#define NOMINMAX
+#endif
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include
+#include
+#endif
+
+#include "orcjit_dylib.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+// Initialize LLVM native target (only once)
+struct LLVMInitializer {
+ LLVMInitializer() {
+llvm::InitializeNativeTarget();
+llvm::InitializeNativeTargetAsmPrinter();
+llvm::InitializeNativeTargetAsmParser();
+ }
+};
+
+static LLVMInitializer llvm_initializer;
+
+class InitFiniPlugin : public llvm::orc::ObjectLinkingLayer::Plugin {
+ ORCJITExecutionSession session_;
+
+ public:
+ explicit InitFiniPlugin(ORCJITExecutionSession session) :
session_(std::move(session)) {}
Review Comment:
already fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986905329
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+ *ctx_addr = symbol;
+}
+ });
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+ORCJITDynamicLibraryObj::~ORCJITDynamicLibraryObj() {
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run section-based deinitializers (.fini_array, .dtors,
.CRT$XT*)
+ // collected by our custom InitFiniPlugin.
+ session_->RunPendingDeinitializers(GetJITDylib());
+#else
+ // macOS: native platform's deinitialize drains __cxa_atexit handlers
+ // (registered during initialization) via the ORC runtime.
+ if (auto err = jit_->deinitialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+}
+
+void ORCJITDynamicLibraryObj::AddObjectFile(const String& path) {
+ // Read object file
+ auto buffer_or_err = llvm::MemoryBuffer::getFile(path.c_str());
+ if (!buffer_or_err) {
+TVM_FFI_THROW(IOError) << "Failed to read object file: " << path;
+ }
+
+ // Add object file to this JITDylib
+ call_llvm(jit_->addObjectFile(*dylib_, std::move(*buffer_or_err)), "Failed
to add object file");
+}
+
+void ORCJITDynamicLibraryObj::SetLinkOrder(const
std::vector& dylibs) {
+ // Rebuild the link order: user-specified libraries first, then the LLJIT
+ // default link order (Main → Platform → ProcessSymbols). Preserving the
+ // default link order is essential — without ProcessSymbols, C++ objects
+ // that need host-process symbols (runtime, libtvm_ffi) would fail to link.
+ link_order_.clear();
+
+ for (auto* lib : dylibs) {
+link_order_.emplace_back(lib,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ }
+ for (auto& kv : jit_->defaultLinkOrder()) {
+link_order_.emplace_back(kv.first, kv.second);
+ }
+
+ // Set the link order in the LLVM JITDylib
+ dylib_->setLinkOrder(link_order_, false);
+}
+
+void* ORCJITDynamicLibraryObj::GetSymbol(const String& name) {
+ // Build search order: this dylib first, then all linked dylibs
+ llvm::orc::JITDylibSearchOrder search_order;
+ search_order.emplace_back(dylib_,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ // Append linked libraries
+ search_order.insert(search_order.end(), link_order_.begin(),
link_order_.end());
+
+ // Look up symbol using the full search order
+ auto symbol_or_err =
+ jit_->getExecutionSession().lookup(search_order,
jit_->mangleAndIntern(name.c_str()));
+
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run initializers collected by our custom InitFiniPlugin.
+ session_->RunPendingInitializers(GetJITDylib());
+#else
+ // macOS: use native platform's init mechanism (handles __mod_init_func
+ // and __cxa_atexit registration).
+ if (auto err = jit_->initialize(*dylib_)) {
Review Comment:
MachOPlatform uses dlopen-like semantics internally. The ORC runtime tracks
initialized JITDylibs with a reference count: first `initialize()` runs
`__mod_init_func` entries and re
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986888594
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.h:
##
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.h
+ * \brief LLVM ORC JIT DynamicLibrary (JITDylib) wrapper
+ */
+#ifndef TVM_FFI_ORCJIT_ORCJIT_DYLIB_H_
+#define TVM_FFI_ORCJIT_ORCJIT_DYLIB_H_
+
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+class ORCJITExecutionSession;
+
+class ORCJITDynamicLibraryObj : public ModuleObj {
+ public:
+ /*!
+ * \brief Constructor
+ * \param session The parent execution session
+ * \param dylib The LLVM JITDylib
+ * \param jit The LLJIT instance
+ * \param name The library name
+ */
+ ORCJITDynamicLibraryObj(ORCJITExecutionSession session, llvm::orc::JITDylib*
dylib,
+ llvm::orc::LLJIT* jit, String name);
+
+ ~ORCJITDynamicLibraryObj();
+
+ const char* kind() const final { return "orcjit"; }
+
+ Optional GetFunction(const String& name) override;
+
+ private:
+ /*!
+ * \brief Add an object file to this library
+ * \param path Path to the object file to load
+ */
+ void AddObjectFile(const String& path);
+
+ /*!
+ * \brief Set the link order for symbol resolution
+ * \param dylibs Vector of libraries to search for symbols (in order)
+ *
+ * When resolving symbols, this library will search in the specified
libraries
+ * in the order provided. This replaces any previous link order.
+ */
+ void SetLinkOrder(const std::vector& dylibs);
+
+ /*!
+ * \brief Look up a symbol in this library
+ * \param name The symbol name to look up
+ * \return Pointer to the symbol, or nullptr if not found
+ */
+ void* GetSymbol(const String& name);
+
+ /*!
+ * \brief Get the underlying LLVM JITDylib
+ * \return Reference to the LLVM JITDylib
+ */
+ llvm::orc::JITDylib& GetJITDylib();
+
+ /*!
+ * \brief Get the name of this library
+ * \return The library name
+ */
+ String GetName() const { return name_; }
+
+ /*! \brief Parent execution session (for lifetime management) */
+ ORCJITExecutionSession session_;
+
+ /*! \brief The LLVM JITDylib */
+ llvm::orc::JITDylib* dylib_;
+
+ /*! \brief The LLJIT instance (for addObjectFile API) */
+ llvm::orc::LLJIT* jit_;
Review Comment:
The session already owns `jit_` as `std::unique_ptr`. The dylib
stores a raw pointer for convenience — avoids going through
`session_->GetLLJIT()` on every `GetSymbol/addObjectFile` call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986876987
##
addons/tvm-ffi-orcjit/examples/quick-start/run.py:
##
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Quick Start Example - Load and call functions from compiled object files.
+
+This script demonstrates how to:
+1. Create an ExecutionSession instance
+2. Create a DynamicLibrary
+3. Load a compiled object file (C++ or pure C)
+4. Get functions by name
+5. Call them like regular Python functions
+
+Usage:
+python run.py # Load C++ object file (add.o)
+python run.py --lang c # Load pure C object file (add_c.o)
+"""
+
+import argparse
+import sys
+from pathlib import Path
+
+# Use the installed package if available; fall back to source tree for
editable dev
+try:
+from tvm_ffi_orcjit import ExecutionSession
+except ImportError:
+sys.path.insert(0, str(Path(__file__).parent.parent.parent / "python"))
+from tvm_ffi_orcjit import ExecutionSession
+
+
+def main() -> int:
+"""Run the quick start example."""
+parser = argparse.ArgumentParser(description="Quick Start Example")
+parser.add_argument(
+"--lang",
+choices=["cpp", "c"],
+default="cpp",
+help="Language variant to load: 'cpp' for add.o (default), 'c' for
add_c.o",
+)
+args = parser.parse_args()
+
+# Select object file based on language choice
+if args.lang == "c":
+obj_file = Path("add_c.o")
+else:
+obj_file = Path("add.o")
+
+if not obj_file.exists():
+print(f"Error: {obj_file} not found!")
+print("Please build with CMake first:")
+print(" cmake -B build && cmake --build build")
+return 1
+
+print(f"Loading object file: {obj_file} (lang={args.lang})")
+
+# Create execution session and dynamic library
+session = ExecutionSession()
+lib = session.create_library()
+lib.add(str(obj_file))
+
+print("Object file loaded successfully\n")
+
+# Get and call the 'add' function
+print("=== Testing add function ===")
+add = lib.get_function("add")
+result = add(10, 20)
+print(f"add(10, 20) = {result}")
+assert result == 30, f"Expected 30, got {result}"
+
+# Get and call the 'multiply' function
+print("\n=== Testing multiply function ===")
+multiply = lib.get_function("multiply")
+result = multiply(7, 6)
+print(f"multiply(7, 6) = {result}")
+assert result == 42, f"Expected 42, got {result}"
+
+# Get and call the 'fibonacci' function
+print("\n=== Testing fibonacci function ===")
+fibonacci = lib.get_function("fibonacci")
+result = fibonacci(10)
+print(f"fibonacci(10) = {result}")
+assert result == 55, f"Expected 55, got {result}"
+
+if args.lang == "cpp":
+# String concatenation only available in C++ variant (uses std::string)
+print("\n=== Testing concat function ===")
+concat = lib.get_function("concat")
+result = concat("Hello, ", "World!")
+print(f"concat('Hello, ', 'World!') = '{result}'")
+assert result == "Hello, World!", f"Expected 'Hello, World!', got
'{result}'"
+# Release the returned String object before JIT module is destroyed
+del result
+del concat
+
+print("\n" + "=" * 50)
+print("All tests passed successfully!")
+print("=" * 50)
+
+# Cleanup: release references in correct order (functions, lib, session)
+del add, multiply, fibonacci
+del lib
+del session
Review Comment:
already fixed
##
addons/tvm-ffi-orcjit/examples/quick-start/run.py:
##
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by appli
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986854632
##
addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/session.py:
##
@@ -0,0 +1,90 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""ORC JIT Execution Session."""
+
+from __future__ import annotations
+
+import sys
+
+from tvm_ffi import Object, register_object
+
+from . import _ffi_api, _lib_dir
+from .dylib import DynamicLibrary
+
+
+def _find_orc_rt_library() -> str | None:
+"""Find the bundled liborc_rt library in the same directory as the
.so/.dll."""
+# Windows: skip ORC runtime entirely. LLVM's COFFPlatform (loaded via
+# ExecutorNativePlatform with liborc_rt) depends on MSVC C++ runtime
symbols
+# that are not available in the JIT environment. On Windows, ORC JIT uses a
+# C-only strategy: JIT objects are compiled as pure C (TVMFFISafeCallType
ABI),
+# avoiding all C++ runtime dependencies (magic statics, RTTI, sized delete,
+# SEH, COMDAT). Our custom InitFiniPlugin handles .CRT$XC*/.CRT$XT*
init/fini
+# sections, and DLLImportDefinitionGenerator resolves __imp_ DLL import
stubs.
+if sys.platform == "win32":
+return None
+patterns = ["liborc_rt*.a"]
+for pattern in patterns:
+for lib_path in _lib_dir.glob(pattern):
+return str(lib_path)
+return None
+
+
+@register_object("orcjit.ExecutionSession")
+class ExecutionSession(Object):
+"""ORC JIT Execution Session.
+
+Manages the LLVM ORC JIT execution environment and creates dynamic
libraries (JITDylibs).
+This is the top-level context for JIT compilation and symbol management.
+
+Examples
+
+>>> session = ExecutionSession()
+>>> lib = session.create_library(name="main")
+>>> lib.add("add.o")
+>>> add_func = lib.get_function("add")
+
+"""
+
+def __init__(self, orc_rt_path: str | None = None) -> None:
+"""Initialize ExecutionSession.
+
+Args:
+orc_rt_path: Optional path to the liborc_rt library. If not
provided,
+it will be automatically discovered using clang.
+
+"""
+if orc_rt_path is None:
+orc_rt_path = _find_orc_rt_library()
+if orc_rt_path is None:
+orc_rt_path = ""
+self.__init_handle_by_constructor__(_ffi_api.ExecutionSession,
orc_rt_path) # type: ignore
+
+def create_library(self, name: str = "") -> DynamicLibrary:
+"""Create a new dynamic library associated with this execution session.
+
+Args:
+name: Optional name for the library. If empty, a unique name will
be generated.
+
+Returns:
+A new DynamicLibrary instance.
+
+"""
+handle = _ffi_api.ExecutionSessionCreateDynamicLibrary(self, name) #
type: ignore
+lib = DynamicLibrary.__new__(DynamicLibrary)
+lib.__move_handle_from__(handle)
Review Comment:
The C++ function orcjit.ExecutionSessionCreateDynamicLibrary returns Module
(the base type), but we need Python's DynamicLibrary which adds add() and
set_link_order() convenience methods. We need current solution to ship a
DynamicLibrary instance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986844961
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
Review Comment:
yes, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-4124919052 > A high-level question is: you used a InitFiniPlugin to handle the ctors/dtors of the loaded object files. And there are some *Platform in the orc_rt (or compiler_rt) will register the handling of ctors/dtors. Maybe add some documentation to explain this (like why on MSVC, we can't depends on the c runtime library, etc). Is it possible to remove the plugin when the orc_rt has better support towards the ctors/dtors? @yaoyaoding here are the status of three platforms - macOS: MachOPlatform (via orc_rt) handles __mod_init_func/__mod_term_func and __cxa_atexit natively. We delegate to jit_->initialize()/deinitialize() — no InitFiniPlugin needed. - Windows: COFFPlatform is unusable (requires MSVC CRT symbols like _CxxThrowException, RTTI vtables — stalled in LLVM for 2+ years). Our plugin handles .CRT$XC*/.CRT$XT* sections. - Linux: ELFNixPlatform not handles ctor/dtor well, before this commit https://github.com/llvm/llvm-project/pull/175981. Would be return to ELFNixPlatform once new llvm release contains this patch commit. So InitFiniPlugin is still needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986715399 ## addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/session.py: ## @@ -0,0 +1,90 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""ORC JIT Execution Session.""" + +from __future__ import annotations + +import sys + +from tvm_ffi import Object, register_object + +from . import _ffi_api, _lib_dir +from .dylib import DynamicLibrary + + +def _find_orc_rt_library() -> str | None: +"""Find the bundled liborc_rt library in the same directory as the .so/.dll.""" +# Windows: skip ORC runtime entirely. LLVM's COFFPlatform (loaded via +# ExecutorNativePlatform with liborc_rt) depends on MSVC C++ runtime symbols +# that are not available in the JIT environment. On Windows, ORC JIT uses a +# C-only strategy: JIT objects are compiled as pure C (TVMFFISafeCallType ABI), +# avoiding all C++ runtime dependencies (magic statics, RTTI, sized delete, +# SEH, COMDAT). Our custom InitFiniPlugin handles .CRT$XC*/.CRT$XT* init/fini +# sections, and DLLImportDefinitionGenerator resolves __imp_ DLL import stubs. +if sys.platform == "win32": +return None +patterns = ["liborc_rt*.a"] Review Comment: this should be part of compiler-rt component of llvm, I learnt it from sources directly, as little docs found actually. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986697465 ## addons/tvm-ffi-orcjit/scripts/install_llvm.ps1: ## @@ -0,0 +1,98 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Install LLVM from conda-forge using micromamba (Windows). +# Usage: powershell -ExecutionPolicy Bypass -File scripts/install_llvm.ps1 [version] +# version defaults to LLVM_VERSION env var, then 22.1.0 + +param( Review Comment: the llvm is statically linked in the tvm-ffi-orcjit, so users do not need to install llvm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986673620 ## addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/session.py: ## @@ -0,0 +1,90 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""ORC JIT Execution Session.""" + +from __future__ import annotations + +import sys + +from tvm_ffi import Object, register_object + +from . import _ffi_api, _lib_dir +from .dylib import DynamicLibrary + + +def _find_orc_rt_library() -> str | None: +"""Find the bundled liborc_rt library in the same directory as the .so/.dll.""" +# Windows: skip ORC runtime entirely. LLVM's COFFPlatform (loaded via +# ExecutorNativePlatform with liborc_rt) depends on MSVC C++ runtime symbols +# that are not available in the JIT environment. On Windows, ORC JIT uses a Review Comment: The current llvm support on COFF ORC runtime is incomplete and has been on hold for more than 2 years. Their CI also has disabled COFF ORC runtime build and tests for 2 years, even though the unit tests exist. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2986565483
##
addons/tvm-ffi-orcjit/examples/quick-start/README.md:
##
@@ -0,0 +1,92 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+# Quick Start Example
+
+Demonstrates basic usage of tvm-ffi-orcjit: compile functions to object files,
+load them into the ORC JIT at runtime, and call them from Python.
+
+## Files
+
+| File | Description |
+| -- | - |
+| `add.cc` | C++ source using `TVM_FFI_DLL_EXPORT_TYPED_FUNC` (automatic type
marshaling, supports `std::string`) |
+| `add_c.c` | Pure C source using `TVMFFISafeCallType` ABI (`__tvm_ffi_`
prefix). No C++ runtime needed. |
+| `run.py` | Python script that loads and calls the compiled functions |
+| `CMakeLists.txt` | Build configuration for both variants |
+
+## Prerequisites
+
+- Python 3.10+, CMake 3.18+, C/C++ compiler
+- `apache-tvm-ffi` and `tvm-ffi-orcjit` packages installed
+
+## Steps
+
+### 1. Build the object files
+
+```bash
+cmake -B build
+cmake --build build
+```
+
+This produces `add.o` (C++) and `add_c.o` (pure C). On Windows, only the C
+variant is built (C++ is not supported for ORC JIT on Windows).
+
+### 2. Run
+
+```bash
+# C++ variant (Linux/macOS)
+python run.py
+
+# Pure C variant (all platforms including Windows)
+python run.py --lang c
+```
+
+## How It Works
+
+**C++ variant** (`add.cc`): Functions are exported with
+`TVM_FFI_DLL_EXPORT_TYPED_FUNC`, which wraps a typed C++ lambda/function into
+TVM-FFI's packed calling convention. Supports C++ types like `std::string`.
+
+**C variant** (`add_c.c`): Functions follow the `TVMFFISafeCallType` ABI
+directly — each function is named `__tvm_ffi_` and manually packs
+arguments/results via `TVMFFIAny`. Zero C++ dependencies, works on all
+platforms including Windows with MSVC or clang-cl.
+
+**Python side** (`run.py`):
+
+```python
+from tvm_ffi_orcjit import ExecutionSession
+
+session = ExecutionSession() # Create ORC JIT session
+lib = session.create_library() # Create a JITDylib
+lib.add("add_c.o") # Load object file into JIT
+add = lib.get_function("add") # Look up symbol
+print(add(10, 20)) # Call like a normal function → 30
+```
+
+## Platform Notes
+
+| Platform | C++ (`add.o`) | C (`add_c.o`) |
+| -- | :-: | :-: |
+| Linux (Clang/GCC) | yes | yes |
+| macOS (Clang/Apple Clang) | yes | yes |
+| Windows (all compilers) | no | yes |
+
+C++ is not supported for ORC JIT on Windows. The
`TVM_FFI_DLL_EXPORT_TYPED_FUNC`
+macro uses `try`/`catch` which requires Itanium exception ABI symbols that the
+MSVC-built host process cannot provide. Use the pure C variant on Windows.
Review Comment:
It is ORC JIT-specific problem on Windows. When building a normal DLL, the
linker resolves exception handling symbols against the CRT. In ORC JIT, we skip
`COFFPlatform` because it requires MSVC CRT symbols (_CxxThrowException, RTTI
vtables, etc.) that are difficult to provide in the JIT environment. Without
`COFFPlatform`, the exception handling symbols from try/catch in
`TVM_FFI_DLL_EXPORT_TYPED_FUNC` can't be resolved. The pure C variant avoids
exception handling entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yaoyaoding commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2977123252
##
addons/tvm-ffi-orcjit/examples/quick-start/run.py:
##
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Quick Start Example - Load and call functions from compiled object files.
+
+This script demonstrates how to:
+1. Create an ExecutionSession instance
+2. Create a DynamicLibrary
+3. Load a compiled object file (C++ or pure C)
+4. Get functions by name
+5. Call them like regular Python functions
+
+Usage:
+python run.py # Load C++ object file (add.o)
+python run.py --lang c # Load pure C object file (add_c.o)
+"""
+
+import argparse
+import sys
+from pathlib import Path
+
+# Use the installed package if available; fall back to source tree for
editable dev
+try:
+from tvm_ffi_orcjit import ExecutionSession
+except ImportError:
+sys.path.insert(0, str(Path(__file__).parent.parent.parent / "python"))
+from tvm_ffi_orcjit import ExecutionSession
+
+
+def main() -> int:
+"""Run the quick start example."""
+parser = argparse.ArgumentParser(description="Quick Start Example")
+parser.add_argument(
+"--lang",
+choices=["cpp", "c"],
+default="cpp",
+help="Language variant to load: 'cpp' for add.o (default), 'c' for
add_c.o",
+)
+args = parser.parse_args()
+
+# Select object file based on language choice
+if args.lang == "c":
+obj_file = Path("add_c.o")
+else:
+obj_file = Path("add.o")
+
+if not obj_file.exists():
+print(f"Error: {obj_file} not found!")
+print("Please build with CMake first:")
+print(" cmake -B build && cmake --build build")
+return 1
+
+print(f"Loading object file: {obj_file} (lang={args.lang})")
+
+# Create execution session and dynamic library
+session = ExecutionSession()
+lib = session.create_library()
+lib.add(str(obj_file))
+
+print("Object file loaded successfully\n")
+
+# Get and call the 'add' function
+print("=== Testing add function ===")
+add = lib.get_function("add")
+result = add(10, 20)
+print(f"add(10, 20) = {result}")
+assert result == 30, f"Expected 30, got {result}"
+
+# Get and call the 'multiply' function
+print("\n=== Testing multiply function ===")
+multiply = lib.get_function("multiply")
+result = multiply(7, 6)
+print(f"multiply(7, 6) = {result}")
+assert result == 42, f"Expected 42, got {result}"
+
+# Get and call the 'fibonacci' function
+print("\n=== Testing fibonacci function ===")
+fibonacci = lib.get_function("fibonacci")
+result = fibonacci(10)
+print(f"fibonacci(10) = {result}")
+assert result == 55, f"Expected 55, got {result}"
+
+if args.lang == "cpp":
+# String concatenation only available in C++ variant (uses std::string)
+print("\n=== Testing concat function ===")
+concat = lib.get_function("concat")
+result = concat("Hello, ", "World!")
+print(f"concat('Hello, ', 'World!') = '{result}'")
+assert result == "Hello, World!", f"Expected 'Hello, World!', got
'{result}'"
+# Release the returned String object before JIT module is destroyed
+del result
+del concat
Review Comment:
it's not very pythonic. Can we add a reference to concat Function in result
String to avoid explicit del order requiement?
##
addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/session.py:
##
@@ -0,0 +1,90 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITI
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2982113833
##
addons/tvm-ffi-orcjit/examples/quick-start/run.py:
##
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Quick Start Example - Load and call functions from compiled object files.
+
+This script demonstrates how to:
+1. Create an ExecutionSession instance
+2. Create a DynamicLibrary
+3. Load a compiled object file (C++ or pure C)
+4. Get functions by name
+5. Call them like regular Python functions
+
+Usage:
+python run.py # Load C++ object file (add.o)
+python run.py --lang c # Load pure C object file (add_c.o)
+"""
+
+import argparse
+import sys
+from pathlib import Path
+
+# Use the installed package if available; fall back to source tree for
editable dev
+try:
+from tvm_ffi_orcjit import ExecutionSession
+except ImportError:
+sys.path.insert(0, str(Path(__file__).parent.parent.parent / "python"))
+from tvm_ffi_orcjit import ExecutionSession
+
+
+def main() -> int:
+"""Run the quick start example."""
+parser = argparse.ArgumentParser(description="Quick Start Example")
+parser.add_argument(
+"--lang",
+choices=["cpp", "c"],
+default="cpp",
+help="Language variant to load: 'cpp' for add.o (default), 'c' for
add_c.o",
+)
+args = parser.parse_args()
+
+# Select object file based on language choice
+if args.lang == "c":
+obj_file = Path("add_c.o")
+else:
+obj_file = Path("add.o")
+
+if not obj_file.exists():
+print(f"Error: {obj_file} not found!")
+print("Please build with CMake first:")
+print(" cmake -B build && cmake --build build")
+return 1
+
+print(f"Loading object file: {obj_file} (lang={args.lang})")
+
+# Create execution session and dynamic library
+session = ExecutionSession()
+lib = session.create_library()
+lib.add(str(obj_file))
+
+print("Object file loaded successfully\n")
+
+# Get and call the 'add' function
+print("=== Testing add function ===")
+add = lib.get_function("add")
+result = add(10, 20)
+print(f"add(10, 20) = {result}")
+assert result == 30, f"Expected 30, got {result}"
+
+# Get and call the 'multiply' function
+print("\n=== Testing multiply function ===")
+multiply = lib.get_function("multiply")
+result = multiply(7, 6)
+print(f"multiply(7, 6) = {result}")
+assert result == 42, f"Expected 42, got {result}"
+
+# Get and call the 'fibonacci' function
+print("\n=== Testing fibonacci function ===")
+fibonacci = lib.get_function("fibonacci")
+result = fibonacci(10)
+print(f"fibonacci(10) = {result}")
+assert result == 55, f"Expected 55, got {result}"
+
+if args.lang == "cpp":
+# String concatenation only available in C++ variant (uses std::string)
+print("\n=== Testing concat function ===")
+concat = lib.get_function("concat")
+result = concat("Hello, ", "World!")
+print(f"concat('Hello, ', 'World!') = '{result}'")
+assert result == "Hello, World!", f"Expected 'Hello, World!', got
'{result}'"
Review Comment:
extracted to the sub function and no more explicit del now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2982106949
##
addons/tvm-ffi-orcjit/pyproject.toml:
##
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.10.0", "apache-tvm-ffi"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm-ffi-orcjit"
+version = "0.1.0"
+description = "Load TVM-FFI exported object files using LLVM ORC JIT v2"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "Apache-2.0" }
+authors = [{ name = "TVM-FFI OrcJIT Contributors" }]
+keywords = ["tvm-ffi", "llvm", "jit", "orcjit"]
+classifiers = [
+ "Development Status :: 3 - Alpha",
+ "Intended Audience :: Developers",
+ "License :: OSI Approved :: Apache Software License",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "Programming Language :: Python :: 3.13",
+ "Programming Language :: C++",
+]
+dependencies = ["apache-tvm-ffi>=0.1.0"]
+
+[project.urls]
+Homepage = "https://github.com/apache/tvm-ffi";
+Repository = "https://github.com/apache/tvm-ffi";
+
+[tool.scikit-build]
+cmake.build-type = "Release"
+wheel.py-api = "py3"
+build-dir = "build"
+build.verbose = true
+editable.rebuild = false
+editable.verbose = true
+wheel.packages = ["python/tvm_ffi_orcjit"]
+wheel.install-dir = "tvm_ffi_orcjit"
+sdist.include = [
+ "/README.md",
+ "/LICENSE",
+ "/pyproject.toml",
+ "/CMakeLists.txt",
+ "/src/**/*.h",
+ "/src/**/*.cc",
+ "/src/**/*.cpp",
+ "/python/**/*.py",
+ "/tools/*.sh",
+ "/tools/*.ps1",
+]
+
+[tool.scikit-build.cmake.define]
+CMAKE_EXPORT_COMPILE_COMMANDS = "ON"
+
+[tool.cibuildwheel]
+# One build per platform is sufficient: the shared library does not link
against
+# Python C API, so wheel.py-api = "py3" produces a version-agnostic wheel.
+build = "cp312-*"
+skip = "*-win32 *-manylinux_i686 *-musllinux*"
+
+[tool.cibuildwheel.linux]
+environment = { LLVM_VERSION = "22.1.0", CMAKE_PREFIX_PATH = "/opt/llvm",
LLVM_CONFIG_PATH = "/opt/llvm/bin/llvm-config" }
Review Comment:
removed and no hardcoded path in pyproject.toml now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2982108816 ## addons/tvm-ffi-orcjit/examples/quick-start/run.py: ## @@ -0,0 +1,122 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Quick Start Example - Load and call functions from compiled object files. + +This script demonstrates how to: +1. Create an ExecutionSession instance +2. Create a DynamicLibrary +3. Load a compiled object file (C++ or pure C) +4. Get functions by name +5. Call them like regular Python functions + +Usage: +python run.py # Load C++ object file (add.o) +python run.py --lang c # Load pure C object file (add_c.o) +""" + +import argparse +import sys +from pathlib import Path + +# Use the installed package if available; fall back to source tree for editable dev +try: +from tvm_ffi_orcjit import ExecutionSession Review Comment: simplified -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2981588148
##
addons/tvm-ffi-orcjit/pyproject.toml:
##
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.10.0", "apache-tvm-ffi"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm-ffi-orcjit"
+version = "0.1.0"
+description = "Load TVM-FFI exported object files using LLVM ORC JIT v2"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "Apache-2.0" }
+authors = [{ name = "TVM-FFI OrcJIT Contributors" }]
+keywords = ["tvm-ffi", "llvm", "jit", "orcjit"]
+classifiers = [
+ "Development Status :: 3 - Alpha",
+ "Intended Audience :: Developers",
+ "License :: OSI Approved :: Apache Software License",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "Programming Language :: Python :: 3.13",
+ "Programming Language :: C++",
+]
+dependencies = ["apache-tvm-ffi>=0.1.0"]
+
+[project.urls]
+Homepage = "https://github.com/apache/tvm-ffi";
+Repository = "https://github.com/apache/tvm-ffi";
+
+[tool.scikit-build]
+cmake.build-type = "Release"
+wheel.py-api = "py3"
+build-dir = "build"
+build.verbose = true
+editable.rebuild = false
+editable.verbose = true
+wheel.packages = ["python/tvm_ffi_orcjit"]
+wheel.install-dir = "tvm_ffi_orcjit"
+sdist.include = [
+ "/README.md",
+ "/LICENSE",
+ "/pyproject.toml",
+ "/CMakeLists.txt",
+ "/src/**/*.h",
+ "/src/**/*.cc",
+ "/src/**/*.cpp",
+ "/python/**/*.py",
+ "/tools/*.sh",
+ "/tools/*.ps1",
+]
+
+[tool.scikit-build.cmake.define]
+CMAKE_EXPORT_COMPILE_COMMANDS = "ON"
+
+[tool.cibuildwheel]
+# One build per platform is sufficient: the shared library does not link
against
+# Python C API, so wheel.py-api = "py3" produces a version-agnostic wheel.
+build = "cp312-*"
+skip = "*-win32 *-manylinux_i686 *-musllinux*"
+
+[tool.cibuildwheel.linux]
+environment = { LLVM_VERSION = "22.1.0", CMAKE_PREFIX_PATH = "/opt/llvm",
LLVM_CONFIG_PATH = "/opt/llvm/bin/llvm-config" }
Review Comment:
cross check if we still need it assuming llvm is installed in conda env
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2981577854 ## addons/tvm-ffi-orcjit/examples/quick-start/run.py: ## @@ -0,0 +1,122 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Quick Start Example - Load and call functions from compiled object files. + +This script demonstrates how to: +1. Create an ExecutionSession instance +2. Create a DynamicLibrary +3. Load a compiled object file (C++ or pure C) +4. Get functions by name +5. Call them like regular Python functions + +Usage: +python run.py # Load C++ object file (add.o) +python run.py --lang c # Load pure C object file (add_c.o) +""" + +import argparse +import sys +from pathlib import Path + +# Use the installed package if available; fall back to source tree for editable dev +try: +from tvm_ffi_orcjit import ExecutionSession Review Comment: keep things simple, let us just assume tvm_ffi_orcjit is installed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2981574795
##
addons/tvm-ffi-orcjit/examples/quick-start/run.py:
##
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Quick Start Example - Load and call functions from compiled object files.
+
+This script demonstrates how to:
+1. Create an ExecutionSession instance
+2. Create a DynamicLibrary
+3. Load a compiled object file (C++ or pure C)
+4. Get functions by name
+5. Call them like regular Python functions
+
+Usage:
+python run.py # Load C++ object file (add.o)
+python run.py --lang c # Load pure C object file (add_c.o)
+"""
+
+import argparse
+import sys
+from pathlib import Path
+
+# Use the installed package if available; fall back to source tree for
editable dev
+try:
+from tvm_ffi_orcjit import ExecutionSession
+except ImportError:
+sys.path.insert(0, str(Path(__file__).parent.parent.parent / "python"))
+from tvm_ffi_orcjit import ExecutionSession
+
+
+def main() -> int:
+"""Run the quick start example."""
+parser = argparse.ArgumentParser(description="Quick Start Example")
+parser.add_argument(
+"--lang",
+choices=["cpp", "c"],
+default="cpp",
+help="Language variant to load: 'cpp' for add.o (default), 'c' for
add_c.o",
+)
+args = parser.parse_args()
+
+# Select object file based on language choice
+if args.lang == "c":
+obj_file = Path("add_c.o")
+else:
+obj_file = Path("add.o")
+
+if not obj_file.exists():
+print(f"Error: {obj_file} not found!")
+print("Please build with CMake first:")
+print(" cmake -B build && cmake --build build")
+return 1
+
+print(f"Loading object file: {obj_file} (lang={args.lang})")
+
+# Create execution session and dynamic library
+session = ExecutionSession()
+lib = session.create_library()
+lib.add(str(obj_file))
+
+print("Object file loaded successfully\n")
+
+# Get and call the 'add' function
+print("=== Testing add function ===")
+add = lib.get_function("add")
+result = add(10, 20)
+print(f"add(10, 20) = {result}")
+assert result == 30, f"Expected 30, got {result}"
+
+# Get and call the 'multiply' function
+print("\n=== Testing multiply function ===")
+multiply = lib.get_function("multiply")
+result = multiply(7, 6)
+print(f"multiply(7, 6) = {result}")
+assert result == 42, f"Expected 42, got {result}"
+
+# Get and call the 'fibonacci' function
+print("\n=== Testing fibonacci function ===")
+fibonacci = lib.get_function("fibonacci")
+result = fibonacci(10)
+print(f"fibonacci(10) = {result}")
+assert result == 55, f"Expected 55, got {result}"
+
+if args.lang == "cpp":
+# String concatenation only available in C++ variant (uses std::string)
+print("\n=== Testing concat function ===")
+concat = lib.get_function("concat")
+result = concat("Hello, ", "World!")
+print(f"concat('Hello, ', 'World!') = '{result}'")
+assert result == "Hello, World!", f"Expected 'Hello, World!', got
'{result}'"
Review Comment:
move check to sub function so we don't need explicit del here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2981541520
##
addons/tvm-ffi-orcjit/tools/install_llvm.ps1:
##
@@ -0,0 +1,98 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Install LLVM from conda-forge using micromamba (Windows).
+# Usage: powershell -ExecutionPolicy Bypass -File tools/install_llvm.ps1
[version]
+# version defaults to LLVM_VERSION env var, then 22.1.0
+
+param(
+[string]$Version = ""
+)
+
+$ErrorActionPreference = "Stop"
+Set-StrictMode -Version Latest
+
+if (-not $Version) {
+$Version = if ($env:LLVM_VERSION) { $env:LLVM_VERSION } else { "22.1.0" }
+}
+$Prefix = if ($env:LLVM_PREFIX) { $env:LLVM_PREFIX } else { "C:\opt\llvm" }
+
+Write-Host "Installing LLVM $Version to $Prefix"
+
+# Install micromamba from GitHub releases (micro.mamba.pm cert expired as of
2026-03)
+$MicromambaExe = "$env:TEMP\micromamba.exe"
+
+if (-not (Test-Path $MicromambaExe)) {
+Write-Host "Downloading micromamba from GitHub releases..."
+$maxRetries = 3
+for ($attempt = 1; $attempt -le $maxRetries; $attempt++) {
+try {
+& curl.exe -sSL -o $MicromambaExe
"https://github.com/mamba-org/micromamba-releases/releases/latest/download/micromamba-win-64.exe";
+if ($LASTEXITCODE -ne 0) { throw "curl failed with exit code
$LASTEXITCODE" }
+break
+} catch {
+Write-Host "Attempt $attempt/$maxRetries failed: $_"
+if ($attempt -eq $maxRetries) { throw }
+Start-Sleep -Seconds 5
+}
+}
+}
+Write-Host "Using micromamba: $MicromambaExe"
+
+# Install LLVM and zlib. No clangdev or compiler-rt on Windows — test objects
+# use C-only strategy compiled with the system compiler (MSVC), and liborc_rt
+# is not used (Windows ORC JIT skips COFFPlatform).
+& $MicromambaExe create -p $Prefix -c conda-forge `
+"llvmdev=$Version" `
+zlib `
+-y
+if ($LASTEXITCODE -ne 0) { throw "micromamba create failed" }
+
+# Build static zstd from source.
Review Comment:
consider bringing to addons/tvm-ffi-orcjit/scripts/install_llvm.ps1,
BTW, if we are using it for gh workflow, likely it can be simplified via
setup miniconda action
https://github.com/apache/tvm/blob/main/.github/actions/setup/action.yml#L21
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_utils.h:
##
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_utils.h
+ * \brief LLVM ORC JIT utils
+ */
+#ifndef TVM_FFI_ORCJIT_ORCJIT_UTILS_H_
+#define TVM_FFI_ORCJIT_ORCJIT_UTILS_H_
+
+#include
+#include
+#include
+
+#include
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+inline void call_llvm(llvm::Error err, const std::string& context_msg = "") {
Review Comment:
minor, CamelCase CallLLVM, also need documentation. To make it lazy, likely
we want to keep TVM_FFI_ORCJIT_LLVM_CALL a macro like TVM_FFI_CUDA_CALL
##
addons/tvm-ffi-orcjit/tests/run_all_tests.py:
##
@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2980112788
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_session.cc:
##
@@ -0,0 +1,664 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_session.cc
+ * \brief LLVM ORC JIT ExecutionSession implementation
+ */
+
+#include "orcjit_session.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+
+#ifdef _WIN32
+#ifndef NOMINMAX
+#define NOMINMAX
+#endif
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include
+#include
+#endif
+
+#include "orcjit_dylib.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+// Initialize LLVM native target (only once)
+struct LLVMInitializer {
+ LLVMInitializer() {
+llvm::InitializeNativeTarget();
+llvm::InitializeNativeTargetAsmPrinter();
+llvm::InitializeNativeTargetAsmParser();
+ }
+};
+
+static LLVMInitializer llvm_initializer;
+
+class InitFiniPlugin : public llvm::orc::ObjectLinkingLayer::Plugin {
+ ORCJITExecutionSession session_;
+
+ public:
+ explicit InitFiniPlugin(ORCJITExecutionSession session) :
session_(std::move(session)) {}
+
+ void modifyPassConfig(llvm::orc::MaterializationResponsibility& MR,
llvm::jitlink::LinkGraph& G,
+llvm::jitlink::PassConfiguration& Config) override {
+auto& jit_dylib = MR.getTargetJITDylib();
+// Mark all init/fini section blocks and their edge targets as live
+// so they survive dead-stripping.
+Config.PrePrunePasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ for (auto& Section : G.sections()) {
+auto section_name = Section.getName();
+// ELF: .init_array*, .fini_array*, .ctors*, .dtors*
+// Mach-O: __DATA,__mod_init_func, __DATA,__mod_term_func
+// COFF: .CRT$XC* (ctors), .CRT$XT* (dtors)
+if (section_name.starts_with(".init_array") ||
section_name.starts_with(".fini_array") ||
+section_name.starts_with(".ctors") ||
section_name.starts_with(".dtors") ||
+section_name == "__DATA,__mod_init_func" || section_name ==
"__DATA,__mod_term_func" ||
+section_name.starts_with(".CRT$XC") ||
section_name.starts_with(".CRT$XT")) {
+ for (auto* Block : Section.blocks()) {
+bool has_live_sym = false;
+for (auto* Sym : G.defined_symbols()) {
+ if (&Sym->getBlock() == Block) {
+Sym->setLive(true);
+has_live_sym = true;
+ }
+}
+// MSVC may emit .CRT$XC* blocks with data but no symbol table
+// entries (static variables in __declspec(allocate) sections).
+// Add an anonymous symbol so the block survives dead-stripping.
+if (!has_live_sym) {
+ G.addAnonymousSymbol(*Block, 0, Block->getSize(), false,
true).setLive(true);
+}
+for (auto& Edge : Block->edges()) {
+ Edge.getTarget().setLive(true);
+}
+ }
+}
+ }
+ return llvm::Error::success();
+});
+#ifdef _WIN32
+// Without COFFPlatform, __ImageBase (used by IMAGE_REL_AMD64_ADDR32NB /
+// Pointer32NB relocations) defaults to 0. This causes all Pointer32 fixups
+// to overflow since JIT addresses don't fit in 32 bits.
+//
+// Fix: set __ImageBase to the lowest block address in the graph after
+// allocation. This makes all intra-graph Pointer32NB offsets small.
+//
+// Also strip .pdata/.xdata edges: SEH unwind data references external
+// handlers (e.g., __CxxFrameHandler3) in DLLs that may be >4GB from
+// __ImageBase. Since we don't call RtlAddFunctionTable, SEH data is
+// unused anyway.
+Config.PostAllocationPasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ // Set __ImageBase to the lowest allocated block address.
+ auto ImageBaseName = G.intern("__ImageBase");
+ llvm::jitlink::Symbol* ImageBase = nullptr;
+ for (auto* Sym : G.external_symbols())
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2980101729
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+ *ctx_addr = symbol;
+}
+ });
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+ORCJITDynamicLibraryObj::~ORCJITDynamicLibraryObj() {
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run section-based deinitializers (.fini_array, .dtors,
.CRT$XT*)
+ // collected by our custom InitFiniPlugin.
+ session_->RunPendingDeinitializers(GetJITDylib());
+#else
+ // macOS: native platform's deinitialize drains __cxa_atexit handlers
+ // (registered during initialization) via the ORC runtime.
+ if (auto err = jit_->deinitialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+}
+
+void ORCJITDynamicLibraryObj::AddObjectFile(const String& path) {
+ // Read object file
+ auto buffer_or_err = llvm::MemoryBuffer::getFile(path.c_str());
+ if (!buffer_or_err) {
+TVM_FFI_THROW(IOError) << "Failed to read object file: " << path;
+ }
+
+ // Add object file to this JITDylib
+ call_llvm(jit_->addObjectFile(*dylib_, std::move(*buffer_or_err)), "Failed
to add object file");
+}
+
+void ORCJITDynamicLibraryObj::SetLinkOrder(const
std::vector& dylibs) {
+ // Rebuild the link order: user-specified libraries first, then the LLJIT
+ // default link order (Main → Platform → ProcessSymbols). Preserving the
+ // default link order is essential — without ProcessSymbols, C++ objects
+ // that need host-process symbols (runtime, libtvm_ffi) would fail to link.
+ link_order_.clear();
+
+ for (auto* lib : dylibs) {
+link_order_.emplace_back(lib,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ }
+ for (auto& kv : jit_->defaultLinkOrder()) {
+link_order_.emplace_back(kv.first, kv.second);
+ }
+
+ // Set the link order in the LLVM JITDylib
+ dylib_->setLinkOrder(link_order_, false);
+}
+
+void* ORCJITDynamicLibraryObj::GetSymbol(const String& name) {
+ // Build search order: this dylib first, then all linked dylibs
+ llvm::orc::JITDylibSearchOrder search_order;
+ search_order.emplace_back(dylib_,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ // Append linked libraries
+ search_order.insert(search_order.end(), link_order_.begin(),
link_order_.end());
+
+ // Look up symbol using the full search order
+ auto symbol_or_err =
+ jit_->getExecutionSession().lookup(search_order,
jit_->mangleAndIntern(name.c_str()));
+
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run initializers collected by our custom InitFiniPlugin.
+ session_->RunPendingInitializers(GetJITDylib());
+#else
+ // macOS: use native platform's init mechanism (handles __mod_init_func
+ // and __cxa_atexit registration).
+ if (auto err = jit_->initialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+ // Convert ExecutorAddr to pointer
+ return symbol_or_err ? symbol_or_err->getAddress().toPtr() : nullptr;
+}
+
+llvm::orc::JITDylib& ORCJITDyna
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2980098853
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
Review Comment:
actually, in `RunPendingInitializers`, the initializers are only called once
and then being removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2980089066
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_session.cc:
##
@@ -0,0 +1,664 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_session.cc
+ * \brief LLVM ORC JIT ExecutionSession implementation
+ */
+
+#include "orcjit_session.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+
+#ifdef _WIN32
+#ifndef NOMINMAX
+#define NOMINMAX
+#endif
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include
+#include
+#endif
+
+#include "orcjit_dylib.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+// Initialize LLVM native target (only once)
+struct LLVMInitializer {
+ LLVMInitializer() {
+llvm::InitializeNativeTarget();
+llvm::InitializeNativeTargetAsmPrinter();
+llvm::InitializeNativeTargetAsmParser();
+ }
+};
+
+static LLVMInitializer llvm_initializer;
+
+class InitFiniPlugin : public llvm::orc::ObjectLinkingLayer::Plugin {
+ ORCJITExecutionSession session_;
+
+ public:
+ explicit InitFiniPlugin(ORCJITExecutionSession session) :
session_(std::move(session)) {}
+
+ void modifyPassConfig(llvm::orc::MaterializationResponsibility& MR,
llvm::jitlink::LinkGraph& G,
+llvm::jitlink::PassConfiguration& Config) override {
+auto& jit_dylib = MR.getTargetJITDylib();
+// Mark all init/fini section blocks and their edge targets as live
+// so they survive dead-stripping.
+Config.PrePrunePasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ for (auto& Section : G.sections()) {
+auto section_name = Section.getName();
+// ELF: .init_array*, .fini_array*, .ctors*, .dtors*
+// Mach-O: __DATA,__mod_init_func, __DATA,__mod_term_func
+// COFF: .CRT$XC* (ctors), .CRT$XT* (dtors)
+if (section_name.starts_with(".init_array") ||
section_name.starts_with(".fini_array") ||
+section_name.starts_with(".ctors") ||
section_name.starts_with(".dtors") ||
+section_name == "__DATA,__mod_init_func" || section_name ==
"__DATA,__mod_term_func" ||
+section_name.starts_with(".CRT$XC") ||
section_name.starts_with(".CRT$XT")) {
+ for (auto* Block : Section.blocks()) {
+bool has_live_sym = false;
+for (auto* Sym : G.defined_symbols()) {
+ if (&Sym->getBlock() == Block) {
+Sym->setLive(true);
+has_live_sym = true;
+ }
+}
+// MSVC may emit .CRT$XC* blocks with data but no symbol table
+// entries (static variables in __declspec(allocate) sections).
+// Add an anonymous symbol so the block survives dead-stripping.
+if (!has_live_sym) {
+ G.addAnonymousSymbol(*Block, 0, Block->getSize(), false,
true).setLive(true);
+}
+for (auto& Edge : Block->edges()) {
+ Edge.getTarget().setLive(true);
+}
+ }
+}
+ }
+ return llvm::Error::success();
+});
+#ifdef _WIN32
+// Without COFFPlatform, __ImageBase (used by IMAGE_REL_AMD64_ADDR32NB /
+// Pointer32NB relocations) defaults to 0. This causes all Pointer32 fixups
+// to overflow since JIT addresses don't fit in 32 bits.
+//
+// Fix: set __ImageBase to the lowest block address in the graph after
+// allocation. This makes all intra-graph Pointer32NB offsets small.
+//
+// Also strip .pdata/.xdata edges: SEH unwind data references external
+// handlers (e.g., __CxxFrameHandler3) in DLLs that may be >4GB from
+// __ImageBase. Since we don't call RtlAddFunctionTable, SEH data is
+// unused anyway.
+Config.PostAllocationPasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ // Set __ImageBase to the lowest allocated block address.
+ auto ImageBaseName = G.intern("__ImageBase");
+ llvm::jitlink::Symbol* ImageBase = nullptr;
+ for (auto* Sym : G.external_symbols())
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
Kathryn-cat commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2978116948
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_session.cc:
##
@@ -0,0 +1,664 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_session.cc
+ * \brief LLVM ORC JIT ExecutionSession implementation
+ */
+
+#include "orcjit_session.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+
+#ifdef _WIN32
+#ifndef NOMINMAX
+#define NOMINMAX
+#endif
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include
+#include
+#endif
+
+#include "orcjit_dylib.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+// Initialize LLVM native target (only once)
+struct LLVMInitializer {
+ LLVMInitializer() {
+llvm::InitializeNativeTarget();
+llvm::InitializeNativeTargetAsmPrinter();
+llvm::InitializeNativeTargetAsmParser();
+ }
+};
+
+static LLVMInitializer llvm_initializer;
+
+class InitFiniPlugin : public llvm::orc::ObjectLinkingLayer::Plugin {
+ ORCJITExecutionSession session_;
+
+ public:
+ explicit InitFiniPlugin(ORCJITExecutionSession session) :
session_(std::move(session)) {}
+
+ void modifyPassConfig(llvm::orc::MaterializationResponsibility& MR,
llvm::jitlink::LinkGraph& G,
+llvm::jitlink::PassConfiguration& Config) override {
+auto& jit_dylib = MR.getTargetJITDylib();
+// Mark all init/fini section blocks and their edge targets as live
+// so they survive dead-stripping.
+Config.PrePrunePasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ for (auto& Section : G.sections()) {
+auto section_name = Section.getName();
+// ELF: .init_array*, .fini_array*, .ctors*, .dtors*
+// Mach-O: __DATA,__mod_init_func, __DATA,__mod_term_func
+// COFF: .CRT$XC* (ctors), .CRT$XT* (dtors)
+if (section_name.starts_with(".init_array") ||
section_name.starts_with(".fini_array") ||
+section_name.starts_with(".ctors") ||
section_name.starts_with(".dtors") ||
+section_name == "__DATA,__mod_init_func" || section_name ==
"__DATA,__mod_term_func" ||
+section_name.starts_with(".CRT$XC") ||
section_name.starts_with(".CRT$XT")) {
+ for (auto* Block : Section.blocks()) {
+bool has_live_sym = false;
+for (auto* Sym : G.defined_symbols()) {
+ if (&Sym->getBlock() == Block) {
+Sym->setLive(true);
+has_live_sym = true;
+ }
+}
+// MSVC may emit .CRT$XC* blocks with data but no symbol table
+// entries (static variables in __declspec(allocate) sections).
+// Add an anonymous symbol so the block survives dead-stripping.
+if (!has_live_sym) {
+ G.addAnonymousSymbol(*Block, 0, Block->getSize(), false,
true).setLive(true);
+}
+for (auto& Edge : Block->edges()) {
+ Edge.getTarget().setLive(true);
+}
+ }
+}
+ }
+ return llvm::Error::success();
+});
+#ifdef _WIN32
+// Without COFFPlatform, __ImageBase (used by IMAGE_REL_AMD64_ADDR32NB /
+// Pointer32NB relocations) defaults to 0. This causes all Pointer32 fixups
+// to overflow since JIT addresses don't fit in 32 bits.
+//
+// Fix: set __ImageBase to the lowest block address in the graph after
+// allocation. This makes all intra-graph Pointer32NB offsets small.
+//
+// Also strip .pdata/.xdata edges: SEH unwind data references external
+// handlers (e.g., __CxxFrameHandler3) in DLLs that may be >4GB from
+// __ImageBase. Since we don't call RtlAddFunctionTable, SEH data is
+// unused anyway.
+Config.PostAllocationPasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ // Set __ImageBase to the lowest allocated block address.
+ auto ImageBaseName = G.intern("__ImageBase");
+ llvm::jitlink::Symbol* ImageBase = nullptr;
+ for (auto* Sym : G.external_symbo
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
Kathryn-cat commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2978099644
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+ *ctx_addr = symbol;
+}
+ });
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+ORCJITDynamicLibraryObj::~ORCJITDynamicLibraryObj() {
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run section-based deinitializers (.fini_array, .dtors,
.CRT$XT*)
+ // collected by our custom InitFiniPlugin.
+ session_->RunPendingDeinitializers(GetJITDylib());
+#else
+ // macOS: native platform's deinitialize drains __cxa_atexit handlers
+ // (registered during initialization) via the ORC runtime.
+ if (auto err = jit_->deinitialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+}
+
+void ORCJITDynamicLibraryObj::AddObjectFile(const String& path) {
+ // Read object file
+ auto buffer_or_err = llvm::MemoryBuffer::getFile(path.c_str());
+ if (!buffer_or_err) {
+TVM_FFI_THROW(IOError) << "Failed to read object file: " << path;
+ }
+
+ // Add object file to this JITDylib
+ call_llvm(jit_->addObjectFile(*dylib_, std::move(*buffer_or_err)), "Failed
to add object file");
+}
+
+void ORCJITDynamicLibraryObj::SetLinkOrder(const
std::vector& dylibs) {
+ // Rebuild the link order: user-specified libraries first, then the LLJIT
+ // default link order (Main → Platform → ProcessSymbols). Preserving the
+ // default link order is essential — without ProcessSymbols, C++ objects
+ // that need host-process symbols (runtime, libtvm_ffi) would fail to link.
+ link_order_.clear();
+
+ for (auto* lib : dylibs) {
+link_order_.emplace_back(lib,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ }
+ for (auto& kv : jit_->defaultLinkOrder()) {
+link_order_.emplace_back(kv.first, kv.second);
+ }
+
+ // Set the link order in the LLVM JITDylib
+ dylib_->setLinkOrder(link_order_, false);
+}
+
+void* ORCJITDynamicLibraryObj::GetSymbol(const String& name) {
+ // Build search order: this dylib first, then all linked dylibs
+ llvm::orc::JITDylibSearchOrder search_order;
+ search_order.emplace_back(dylib_,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ // Append linked libraries
+ search_order.insert(search_order.end(), link_order_.begin(),
link_order_.end());
+
+ // Look up symbol using the full search order
+ auto symbol_or_err =
+ jit_->getExecutionSession().lookup(search_order,
jit_->mangleAndIntern(name.c_str()));
+
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run initializers collected by our custom InitFiniPlugin.
+ session_->RunPendingInitializers(GetJITDylib());
+#else
+ // macOS: use native platform's init mechanism (handles __mod_init_func
+ // and __cxa_atexit registration).
+ if (auto err = jit_->initialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+ // Convert ExecutorAddr to pointer
+ return symbol_or_err ? symbol_or_err->getAddress().toPtr() : nullptr;
+}
+
+llvm::orc::JITDylib& ORCJ
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
Kathryn-cat commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2978096242
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
Review Comment:
GetSymbol calls seem repetitive. since each GetSymbol call will trigger the
initializers, it will be init too many times.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
Kathryn-cat commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2978066849
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include "orcjit_dylib.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "orcjit_session.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibraryObj::ORCJITDynamicLibraryObj(ORCJITExecutionSession
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+*ctx_addr = this;
+ }
+ Module::VisitContextSymbols([this](const ffi::String& name, void* symbol) {
+if (void** ctx_addr =
reinterpret_cast(GetSymbol(ffi::symbol::tvm_ffi_library_ctx))) {
+ *ctx_addr = symbol;
+}
+ });
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+ORCJITDynamicLibraryObj::~ORCJITDynamicLibraryObj() {
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run section-based deinitializers (.fini_array, .dtors,
.CRT$XT*)
+ // collected by our custom InitFiniPlugin.
+ session_->RunPendingDeinitializers(GetJITDylib());
+#else
+ // macOS: native platform's deinitialize drains __cxa_atexit handlers
+ // (registered during initialization) via the ORC runtime.
+ if (auto err = jit_->deinitialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+}
+
+void ORCJITDynamicLibraryObj::AddObjectFile(const String& path) {
+ // Read object file
+ auto buffer_or_err = llvm::MemoryBuffer::getFile(path.c_str());
+ if (!buffer_or_err) {
+TVM_FFI_THROW(IOError) << "Failed to read object file: " << path;
+ }
+
+ // Add object file to this JITDylib
+ call_llvm(jit_->addObjectFile(*dylib_, std::move(*buffer_or_err)), "Failed
to add object file");
+}
+
+void ORCJITDynamicLibraryObj::SetLinkOrder(const
std::vector& dylibs) {
+ // Rebuild the link order: user-specified libraries first, then the LLJIT
+ // default link order (Main → Platform → ProcessSymbols). Preserving the
+ // default link order is essential — without ProcessSymbols, C++ objects
+ // that need host-process symbols (runtime, libtvm_ffi) would fail to link.
+ link_order_.clear();
+
+ for (auto* lib : dylibs) {
+link_order_.emplace_back(lib,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ }
+ for (auto& kv : jit_->defaultLinkOrder()) {
+link_order_.emplace_back(kv.first, kv.second);
+ }
+
+ // Set the link order in the LLVM JITDylib
+ dylib_->setLinkOrder(link_order_, false);
+}
+
+void* ORCJITDynamicLibraryObj::GetSymbol(const String& name) {
+ // Build search order: this dylib first, then all linked dylibs
+ llvm::orc::JITDylibSearchOrder search_order;
+ search_order.emplace_back(dylib_,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols);
+ // Append linked libraries
+ search_order.insert(search_order.end(), link_order_.begin(),
link_order_.end());
+
+ // Look up symbol using the full search order
+ auto symbol_or_err =
+ jit_->getExecutionSession().lookup(search_order,
jit_->mangleAndIntern(name.c_str()));
+
+#if defined(__linux__) || defined(_WIN32)
+ // Linux/Windows: run initializers collected by our custom InitFiniPlugin.
+ session_->RunPendingInitializers(GetJITDylib());
+#else
+ // macOS: use native platform's init mechanism (handles __mod_init_func
+ // and __cxa_atexit registration).
+ if (auto err = jit_->initialize(*dylib_)) {
+llvm::consumeError(std::move(err));
+ }
+#endif
+ // Convert ExecutorAddr to pointer
+ return symbol_or_err ? symbol_or_err->getAddress().toPtr() : nullptr;
+}
+
+llvm::orc::JITDylib& ORCJ
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
Kathryn-cat commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2978018686
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_session.cc:
##
@@ -0,0 +1,664 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_session.cc
+ * \brief LLVM ORC JIT ExecutionSession implementation
+ */
+
+#include "orcjit_session.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+
+#ifdef _WIN32
+#ifndef NOMINMAX
+#define NOMINMAX
+#endif
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include
+#include
+#endif
+
+#include "orcjit_dylib.h"
+#include "orcjit_utils.h"
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+// Initialize LLVM native target (only once)
+struct LLVMInitializer {
+ LLVMInitializer() {
+llvm::InitializeNativeTarget();
+llvm::InitializeNativeTargetAsmPrinter();
+llvm::InitializeNativeTargetAsmParser();
+ }
+};
+
+static LLVMInitializer llvm_initializer;
+
+class InitFiniPlugin : public llvm::orc::ObjectLinkingLayer::Plugin {
+ ORCJITExecutionSession session_;
+
+ public:
+ explicit InitFiniPlugin(ORCJITExecutionSession session) :
session_(std::move(session)) {}
+
+ void modifyPassConfig(llvm::orc::MaterializationResponsibility& MR,
llvm::jitlink::LinkGraph& G,
+llvm::jitlink::PassConfiguration& Config) override {
+auto& jit_dylib = MR.getTargetJITDylib();
+// Mark all init/fini section blocks and their edge targets as live
+// so they survive dead-stripping.
+Config.PrePrunePasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ for (auto& Section : G.sections()) {
+auto section_name = Section.getName();
+// ELF: .init_array*, .fini_array*, .ctors*, .dtors*
+// Mach-O: __DATA,__mod_init_func, __DATA,__mod_term_func
+// COFF: .CRT$XC* (ctors), .CRT$XT* (dtors)
+if (section_name.starts_with(".init_array") ||
section_name.starts_with(".fini_array") ||
+section_name.starts_with(".ctors") ||
section_name.starts_with(".dtors") ||
+section_name == "__DATA,__mod_init_func" || section_name ==
"__DATA,__mod_term_func" ||
+section_name.starts_with(".CRT$XC") ||
section_name.starts_with(".CRT$XT")) {
+ for (auto* Block : Section.blocks()) {
+bool has_live_sym = false;
+for (auto* Sym : G.defined_symbols()) {
+ if (&Sym->getBlock() == Block) {
+Sym->setLive(true);
+has_live_sym = true;
+ }
+}
+// MSVC may emit .CRT$XC* blocks with data but no symbol table
+// entries (static variables in __declspec(allocate) sections).
+// Add an anonymous symbol so the block survives dead-stripping.
+if (!has_live_sym) {
+ G.addAnonymousSymbol(*Block, 0, Block->getSize(), false,
true).setLive(true);
+}
+for (auto& Edge : Block->edges()) {
+ Edge.getTarget().setLive(true);
+}
+ }
+}
+ }
+ return llvm::Error::success();
+});
+#ifdef _WIN32
+// Without COFFPlatform, __ImageBase (used by IMAGE_REL_AMD64_ADDR32NB /
+// Pointer32NB relocations) defaults to 0. This causes all Pointer32 fixups
+// to overflow since JIT addresses don't fit in 32 bits.
+//
+// Fix: set __ImageBase to the lowest block address in the graph after
+// allocation. This makes all intra-graph Pointer32NB offsets small.
+//
+// Also strip .pdata/.xdata edges: SEH unwind data references external
+// handlers (e.g., __CxxFrameHandler3) in DLLs that may be >4GB from
+// __ImageBase. Since we don't call RtlAddFunctionTable, SEH data is
+// unused anyway.
+Config.PostAllocationPasses.emplace_back([](llvm::jitlink::LinkGraph& G) {
+ // Set __ImageBase to the lowest allocated block address.
+ auto ImageBaseName = G.intern("__ImageBase");
+ llvm::jitlink::Symbol* ImageBase = nullptr;
+ for (auto* Sym : G.external_symbo
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yaoyaoding commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-3690272617 > What's the status of it? Happy to get it in @cyx-6 is working on this PR right now and currently working on the support of cuda kernel with the orcjitv2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
junrushao commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-3685195840 This PR has been outstanding for a while, and looks indeed super interesting! I don't really have the expertise to review, unfortunately. @yaoyaoding @cyx-6 What's the status of it? Happy to get it in -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
cyx-6 commented on code in PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2547696543 ## addons/tvm-ffi-orcjit/MANIFEST.in: ## @@ -0,0 +1,7 @@ +include README.md +include LICENSE +include pyproject.toml +include CMakeLists.txt +recursive-include include *.h Review Comment: removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2541991838
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -168,34 +141,24 @@ static void RegisterOrcJITFunctions() {
namespace refl = tvm::ffi::reflection;
refl::GlobalDef()
- .def("orcjit.CreateExecutionSession",
- []() -> ORCJITExecutionSession { return
ORCJITExecutionSession::Create(); })
- .def("orcjit.SessionCreateDynamicLibrary",
- [](ORCJITExecutionSession session, String name) -> ObjectRef {
- auto session_obj = GetObjectPtr(
-
const_cast(session.as()));
- return ObjectRef(session_obj->CreateDynamicLibrary(name));
+ .def("orcjit.ExecutionSession", ORCJITExecutionSession::Create)
+ .def("orcjit.ExecutionSessionCreateDynamicLibrary",
+ [](const ORCJITExecutionSession& session, const String& name) ->
ObjectRef {
+ return session->CreateDynamicLibrary(name);
})
.def("orcjit.DynamicLibraryAdd",
Review Comment:
because dynamic library is only a Module and is not an explicit subclass,
let us avoid passing these function via def, instead we can overload
GetFunction in ORCJITDynamicLibrary to return set link order and AddFile etc
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.h:
##
@@ -38,15 +40,7 @@ namespace orcjit {
class ORCJITExecutionSessionObj;
-/*!
- * \brief DynamicLibrary wrapper for LLVM ORC JIT v2 JITDylib
- *
- * This class wraps an LLVM JITDylib and provides functionality to:
- * - Load object files
- * - Link against other dynamic libraries
- * - Look up symbols
- */
-class ORCJITDynamicLibrary : public Object {
+class DynamicLibraryObj : public ModuleObj {
Review Comment:
Keep the name ORCJITDynamicLibraryObj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-3516594099 I think most likely they can evolve independently for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
yzh119 commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-3514910602 One general question, how do we plan to manage the versions of these add ons? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2512405385
##
addons/tvm-ffi-orcjit/MANIFEST.in:
##
@@ -0,0 +1,7 @@
+include README.md
+include LICENSE
+include pyproject.toml
+include CMakeLists.txt
+recursive-include include *.h
Review Comment:
manifest is not needed and only need in pyproject
##
addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/dylib.py:
##
@@ -0,0 +1,133 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""ORC JIT Dynamic Library."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+from tvm_ffi import Function, get_global_func
+from tvm_ffi._ffi_api import ModuleGetFunction
+
+if TYPE_CHECKING:
+from .session import ExecutionSession
+
+
+class DynamicLibrary:
Review Comment:
Make it wrap ffi.Module, so we can directly do lib.foo
##
addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/dylib.py:
##
@@ -0,0 +1,133 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""ORC JIT Dynamic Library."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+from tvm_ffi import Function, get_global_func
+from tvm_ffi._ffi_api import ModuleGetFunction
+
+if TYPE_CHECKING:
+from .session import ExecutionSession
+
+
+class DynamicLibrary:
+"""ORC JIT Dynamic Library (JITDylib).
+
+Represents a collection of symbols that can be loaded from object files
and linked
+against other dynamic libraries. Supports JIT compilation and symbol
resolution.
+
+Examples
+
+>>> session = create_session()
+>>> lib = session.create_library()
+>>> lib.add("add.o")
+>>> lib.add("multiply.o")
+>>> add_func = lib.get_function("add")
+>>> result = add_func(1, 2)
+
+"""
+
+def __init__(self, handle: Any, session: ExecutionSession) -> None:
+"""Initialize DynamicLibrary from a handle.
+
+Parameters
+--
+handle : object
+The underlying C++ ORCJITDynamicLibrary object.
+session : ExecutionSession
+The parent execution session (kept alive for the library's
lifetime).
+
+"""
+self._handle = handle
+self._session = session # Keep session alive
+self._add_func = get_global_func("orcjit.DynamicLibraryAdd")
+self._link_func = get_global_func("orcjit.DynamicLibraryLinkAgainst")
+self._to_module_func = get_global_func("orcjit.DynamicLibraryToModule")
+
+def add(self, object_file: str | Path) -> None:
+"""Add an object file to this dynamic library.
+
+Parameters
+--
+object_file : str or Path
+Path to the object file to load.
+
+Examples
+
+>>> lib.add("add.o")
+>>> lib.add(Path("multiply.o"))
+
+"""
+if isinstance(object_file, Path):
+object_file = str(object_file)
+self._add_func(self._handle, object_file)
+
+def link_against(self, *libraries: DynamicLibrary) -> None:
Review Comment:
lib.set_link_order(list[libraries])
##
addons/tvm-ffi-orcjit/python/tvm_ffi_orcjit/dylib.py:
##
@@ -0,0 +1,133 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright owners
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
tqchen commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2512381001
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file orcjit_dylib.cc
+ * \brief LLVM ORC JIT DynamicLibrary implementation
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+namespace tvm {
+namespace ffi {
+namespace orcjit {
+
+ORCJITDynamicLibrary::ORCJITDynamicLibrary(ObjectPtr
session,
+ llvm::orc::JITDylib* dylib,
llvm::orc::LLJIT* jit,
+ String name)
+: session_(std::move(session)), dylib_(dylib), jit_(jit),
name_(std::move(name)) {
+ TVM_FFI_CHECK(dylib_ != nullptr, ValueError) << "JITDylib cannot be null";
+ TVM_FFI_CHECK(jit_ != nullptr, ValueError) << "LLJIT cannot be null";
+}
+
+void ORCJITDynamicLibrary::AddObjectFile(const String& path) {
+ // Read object file
+ auto buffer_or_err = llvm::MemoryBuffer::getFile(path.c_str());
+ if (!buffer_or_err) {
+TVM_FFI_THROW(IOError) << "Failed to read object file: " << path;
+ }
+
+ // Add object file to this JITDylib
+ auto err = jit_->addObjectFile(*dylib_, std::move(*buffer_or_err));
+ if (err) {
+std::string err_msg;
+llvm::handleAllErrors(std::move(err),
+ [&](const llvm::ErrorInfoBase& eib) { err_msg =
eib.message(); });
+TVM_FFI_THROW(ValueError) << "Failed to add object file '" << path << "':
" << err_msg;
+ }
+}
+
+void ORCJITDynamicLibrary::LinkAgainst(const ORCJITDynamicLibrary& other) {
+ // Set up link order: this dylib should search in other dylib
+ llvm::orc::JITDylibSearchOrder search_order;
+ search_order.push_back({other.dylib_,
llvm::orc::JITDylibLookupFlags::MatchAllSymbols});
+
+ dylib_->setLinkOrder(search_order, false);
+}
+
+void* ORCJITDynamicLibrary::GetSymbol(const String& name) {
+ // Look up symbol
+ auto symbol_or_err =
+ jit_->getExecutionSession().lookup({dylib_},
jit_->mangleAndIntern(name.c_str()));
+ if (!symbol_or_err) {
+auto err = symbol_or_err.takeError();
+std::string err_msg;
+llvm::handleAllErrors(std::move(err),
+ [&](const llvm::ErrorInfoBase& eib) { err_msg =
eib.message(); });
+TVM_FFI_THROW(ValueError) << "Failed to find symbol '" << name << "': " <<
err_msg;
+ }
+
+ // Convert ExecutorAddr to pointer
+ return symbol_or_err->getAddress().toPtr();
+}
+
+llvm::orc::JITDylib& ORCJITDynamicLibrary::GetJITDylib() {
+ TVM_FFI_CHECK(dylib_ != nullptr, InternalError) << "JITDylib is null";
+ return *dylib_;
+}
+
+//-
+// Module wrapper for DynamicLibrary
+//-
+
+class DynamicLibraryModuleObj : public ModuleObj {
+ public:
+ explicit DynamicLibraryModuleObj(ObjectPtr dylib)
+ : dylib_(std::move(dylib)) {}
+
+ const char* kind() const final { return "orcjit_dynamic_library"; }
+
+ Optional GetFunction(const String& name) override {
+// TVM-FFI exports have __tvm_ffi_ prefix
+std::string symbol_name = "__tvm_ffi_" + std::string(name);
+
+// Try to get the symbol - return NullOpt if not found
+void* symbol = nullptr;
+try {
+ symbol = dylib_->GetSymbol(symbol_name);
+} catch (const Error& e) {
+ // Symbol not found
+ return Optional();
+}
+
+// Wrap C function pointer as tvm-ffi Function
+using TVMFFISafeCallType =
+int (*)(void* handle, const TVMFFIAny* args, int32_t num_args,
TVMFFIAny* rv);
+auto c_func = reinterpret_cast(symbol);
+
+return Function::FromPacked([c_func, name](PackedArgs args, Any* rv) {
Review Comment:
https://github.com/apache/tvm-ffi/blob/main/src/ffi/extra/library_module.cc#L53
##
addons/tvm-ffi-orcjit/src/ffi/orcjit_dylib.cc:
##
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownershi
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
gemini-code-assist[bot] commented on code in PR #254:
URL: https://github.com/apache/tvm-ffi/pull/254#discussion_r2512328147
##
addons/tvm-ffi-orcjit/CMakeLists.txt:
##
@@ -0,0 +1,114 @@
+cmake_minimum_required(VERSION 3.18)
+project(
+ tvm_ffi_orcjit
+ VERSION 0.1.0
+ LANGUAGES C CXX
+)
+
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
+set(CMAKE_POSITION_INDEPENDENT_CODE ON)
+
+# Find dependencies
+find_package(LLVM REQUIRED CONFIG)
+message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
+message(STATUS "Using LLVMConfig.cmake in: ${LLVM_DIR}")
+
+# Add LLVM definitions and include directories early
+separate_arguments(LLVM_DEFINITIONS_LIST NATIVE_COMMAND ${LLVM_DEFINITIONS})
+add_definitions(${LLVM_DEFINITIONS_LIST})
+include_directories(${LLVM_INCLUDE_DIRS})
+
+# Find tvm-ffi Look for tvm-ffi in the parent repository first, then system
+set(TVM_FFI_ROOT
+"${CMAKE_CURRENT_SOURCE_DIR}/../.."
+CACHE PATH "Path to tvm-ffi"
+)
+
+if (EXISTS "${TVM_FFI_ROOT}/include/tvm/ffi/c_api.h")
+ message(STATUS "Using tvm-ffi from: ${TVM_FFI_ROOT}")
+
+ # Determine the library extension based on platform
+ if (APPLE)
+set(TVM_FFI_LIB_EXT "dylib")
+ elseif (WIN32)
+set(TVM_FFI_LIB_EXT "dll")
+ else ()
+set(TVM_FFI_LIB_EXT "so")
+ endif ()
+
+ # Create imported target for tvm_ffi
+ add_library(tvm_ffi SHARED IMPORTED)
+ set_target_properties(
+tvm_ffi
+PROPERTIES IMPORTED_LOCATION
"${TVM_FFI_ROOT}/build/lib/libtvm_ffi.${TVM_FFI_LIB_EXT}"
+ INTERFACE_INCLUDE_DIRECTORIES
+
"${TVM_FFI_ROOT}/include;${TVM_FFI_ROOT}/3rdparty/dlpack/include"
+ )
+
+ # Set include directories (including src for internal headers)
+ include_directories(${TVM_FFI_ROOT}/include)
+ include_directories(${TVM_FFI_ROOT}/src)
+ include_directories(${TVM_FFI_ROOT}/3rdparty/dlpack/include)
+else ()
+ message(STATUS "Looking for system tvm-ffi")
+ find_package(tvm-ffi REQUIRED CONFIG)
+endif ()
+
+# LLVM components needed for ORC JIT v2
+llvm_map_components_to_libnames(LLVM_LIBS Core OrcJIT Support native)
+
+# Filter out non-existent targets
+set(LLVM_LIBS_FILTERED)
+foreach (lib ${LLVM_LIBS})
+ if (TARGET ${lib})
+list(APPEND LLVM_LIBS_FILTERED ${lib})
+ else ()
+message(STATUS "Skipping non-existent LLVM target: ${lib}")
+ endif ()
+endforeach ()
+set(LLVM_LIBS ${LLVM_LIBS_FILTERED})
+
+# Source files
+set(SOURCES src/ffi/orcjit_session.cc src/ffi/orcjit_dylib.cc)
+
+# Build shared library
+add_library(tvm_ffi_orcjit SHARED ${SOURCES})
+
+target_include_directories(
+ tvm_ffi_orcjit
+ PUBLIC $
$
+ PRIVATE ${LLVM_INCLUDE_DIRS}
+)
+
+target_link_libraries(
+ tvm_ffi_orcjit
+ PUBLIC tvm_ffi
+ PRIVATE LLVM
Review Comment:
The `find_package(LLVM ... CONFIG)` command does not create a single `LLVM`
target. Instead, `llvm_map_components_to_libnames` populates the `LLVM_LIBS`
variable with a list of component library targets. You should link against
`${LLVM_LIBS}`.
```
PRIVATE ${LLVM_LIBS}
```
##
addons/tvm-ffi-orcjit/CMakeLists.txt:
##
@@ -0,0 +1,114 @@
+cmake_minimum_required(VERSION 3.18)
+project(
+ tvm_ffi_orcjit
+ VERSION 0.1.0
+ LANGUAGES C CXX
+)
+
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
+set(CMAKE_POSITION_INDEPENDENT_CODE ON)
+
+# Find dependencies
+find_package(LLVM REQUIRED CONFIG)
+message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
+message(STATUS "Using LLVMConfig.cmake in: ${LLVM_DIR}")
+
+# Add LLVM definitions and include directories early
+separate_arguments(LLVM_DEFINITIONS_LIST NATIVE_COMMAND ${LLVM_DEFINITIONS})
+add_definitions(${LLVM_DEFINITIONS_LIST})
Review Comment:
These `add_definitions` are redundant as `target_compile_definitions` is
used for the `tvm_ffi_orcjit` target on line 93. It's better to use
`target_compile_definitions` for target-specific definitions. I suggest
removing these lines to avoid redundancy.
##
addons/tvm-ffi-orcjit/README.md:
##
@@ -0,0 +1,230 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+# TVM-FFI OrcJIT
+
+A Python package that enables dynamic loading of TVM-FFI exported object files
(`.o`) using LLVM ORC JIT v2.
+
+## Features
+
+- **Dynamic Loading**: Load compiled object files at runtime using LLVM's ORC
JIT v2
+- **Incremental Loading**: Add multiple object files to the same loader
instance
+- **TVM-FFI Integration**: Seamlessly works with TVM-FFI's stable C ABI
+- **Python API**: Simple Pythonic interface for loading and calling compiled
functions
+- **Standalone Package**: Works alongside apache-tvm-ffi without conflicts
+
+## Installation
+
+### Prerequisites
+
+- Python 3.8+
+- CMake 3.18+
+- LLVM 14+ (with ORC JIT support)
+- Ninja build system (recommende
Re: [PR] [OrcJIT] Add LLVM ORC JIT v2 dynamic object loader addon [tvm-ffi]
gemini-code-assist[bot] commented on PR #254: URL: https://github.com/apache/tvm-ffi/pull/254#issuecomment-3514353343 ## Summary of Changes Hello @yaoyaoding, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant new capability to TVM-FFI by integrating LLVM's ORC JIT v2. This integration enables the dynamic loading and execution of C++ object files directly from Python, offering a flexible and efficient way to extend TVM-FFI's functionality at runtime. The new addon provides a clear Python API for managing JIT compilation contexts and dynamic libraries, making it easier for developers to incorporate custom C++ logic without requiring a full recompile of their applications. ### Highlights * **New Addon Package**: Introduces `tvm-ffi-orcjit`, a new addon package designed to enable dynamic loading of TVM-FFI exported object files (.o) at runtime. * **LLVM ORC JIT v2 Integration**: Leverages LLVM's ORC JIT v2 engine for Just-In-Time compilation and execution of C++ code, providing a powerful and flexible runtime environment. * **Python API**: Provides a Pythonic API centered around `ExecutionSession` (JIT compilation context) and `DynamicLibrary` (manages object files and symbols) for seamless interaction. * **Incremental Loading**: Supports incremental loading, allowing multiple object files to be added to the same `DynamicLibrary` instance, with functions from all loaded files remaining accessible. * **Comprehensive Examples and Tests**: Includes a quick-start example demonstrating basic usage and a robust test suite covering core functionalities, including symbol resolution and conflict handling. Ignored Files * **Ignored by pattern:** `.github/workflows/**` (3) * .github/workflows/orcjit-publish.yml * .github/workflows/orcjit-tests.yml * .github/workflows/tvm-ffi-orcjit/ci_test.yml Using Gemini Code Assist The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either `/gemini ` or `@gemini-code-assist `. Below is a summary of the supported commands on the current page. Feature | Command | Description --- | --- | --- Code Review | `/gemini review` | Performs a code review for the current pull request in its current state. Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state. Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments. Help | `/gemini help` | Displays a list of available commands. Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG). You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist). [^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastruc
