This is an automated email from the ASF dual-hosted git repository.

dbtsai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-connect-rust.git


The following commit(s) were added to refs/heads/master by this push:
     new cdf3aa8  [SPARK-60917][CONNECT][FOLLOWUP] Test aarch64 SHA builds in CI
cdf3aa8 is described below

commit cdf3aa887a5fb69b06b06f0ec1a4d6a3bf7fea3e
Author: DB Tsai <[email protected]>
AuthorDate: Thu Nov 5 11:43:13 2026 -0800

    [SPARK-60917][CONNECT][FOLLOWUP] Test aarch64 SHA builds in CI
    
    ### What changes were proposed in this pull request?
    
    Follow-up to #155 and 
[SPARK-60917](https://issues.apache.org/jira/browse/SPARK-60917).
    
    Add a focused `ubuntu-24.04-arm` job to Rust CI. It installs 
`build-essential` and `protobuf-compiler`, then runs locked release transport 
unit tests with the explicit `aarch64-unknown-linux-gnu` target. Its 
Cargo/target cache includes the runner architecture.
    
    Document the C compiler/assembler and archiver requirement for non-Windows 
aarch64 source builds, including target tools for cross-compilation, and link 
it from the Python and contributor build instructions. Add a short comment that 
artifact upload retries must rebuild the consumed request stream.
    
    ### Why are the changes needed?
    
    #155 enabled `sha2-asm` on non-Windows aarch64, but the existing PR jobs 
run on x86_64. A native ARM job compiles the assembly dependency and runs the 
transport tests on each PR and master/maintenance-branch push, exposing ARM 
build failures before the release workflow needs the dependency.
    
    ### Does this PR introduce _any_ user-facing change?
    
    The source-build documentation now describes the required target C tools. 
Prebuilt Python wheel installation requires no C toolchain. There is no API or 
upload behavior change.
    
    ### How was this patch tested?
    
    - `actionlint` 1.7.12 passed for `.github/workflows/rust.yml`.
    - On macOS aarch64: 107 transport and 414 Rust API release unit tests 
passed. All client integration targets compiled with and without `wasm-udf`; 
formatting and whitespace checks passed.
    - 539 offline/mock Python tests passed, with eight live-server tests 
skipped in that invocation. Focused native Arrow and official 
creation/collection parity checks passed against Spark 4.3.0. API introspection 
found only the intentionally absent `Column.to_plan`.
    - Workspace Clippy reports existing errors in unchanged API/Python/test 
files; none point into the files changed by this PR.
    - The new [native Linux ARM 
job](https://github.com/apache/spark-connect-rust/actions/runs/42088937091/job/127461503950)
 passed on this PR: `sha2-asm` 0.6.4 compiled and 107 transport tests passed. 
The release workflow retains its separate wheel builds.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: OpenAI Codex (application version unavailable).
    
    This pull request and its description were written by Isaac.
    
    Closes #163 from dbtsai/codex/arm-sha-ci.
    
    Authored-by: DB Tsai <[email protected]>
    Signed-off-by: DB Tsai <[email protected]>
    Co-authored-by: Isaac <[email protected]>
---
 .github/workflows/rust.yml              | 31 +++++++++++++++++++++++++++++++
 crates/spark-connect-core/src/client.rs |  2 ++
 docs/contributing.md                    |  4 +++-
 docs/installation.md                    |  9 +++++++++
 4 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml
index a064179..82c00df 100644
--- a/.github/workflows/rust.yml
+++ b/.github/workflows/rust.yml
@@ -89,6 +89,37 @@ jobs:
         # Enforce the "100% coverage, no followups" rule
         run: bash scripts/audit_no_stubs.sh
 
+  aarch64-core:
+    name: Core transport tests (aarch64 Linux)
+    runs-on: ubuntu-24.04-arm
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install Rust toolchain
+        run: |
+          rustup toolchain install stable --profile minimal
+          rustup default stable
+
+      - name: Cache cargo & target
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/.cargo/registry
+            ~/.cargo/git
+            target
+          key: ${{ runner.os }}-${{ runner.arch }}-cargo-aarch64-${{ 
hashFiles('**/Cargo.lock') }}
+          restore-keys: ${{ runner.os }}-${{ runner.arch }}-cargo-aarch64-
+
+      - name: Install system dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y build-essential protobuf-compiler
+
+      - name: Test native aarch64 transport
+        # Compile sha2-asm with the target C toolchain before release builds 
need it.
+        run: cargo test --locked -p apache-spark-connect-core --release --lib 
--target aarch64-unknown-linux-gnu
+
   wasm-udf:
     name: WASM UDF feature
     runs-on: ubuntu-latest
diff --git a/crates/spark-connect-core/src/client.rs 
b/crates/spark-connect-core/src/client.rs
index 03b94af..f0e03df 100644
--- a/crates/spark-connect-core/src/client.rs
+++ b/crates/spark-connect-core/src/client.rs
@@ -888,6 +888,8 @@ impl SparkConnectClient {
 
     /// Send artifact frames and validate the response.
     /// Cache uploads also require an acknowledgement for every requested 
artifact.
+    /// The stream is consumed once; retries must rebuild the frames and 
restart
+    /// AddArtifacts rather than resume a partially consumed stream.
     async fn upload_artifact_requests(
         &self,
         requests: impl futures::Stream<Item = 
spark_connect_proto::AddArtifactsRequest> + Send + 'static,
diff --git a/docs/contributing.md b/docs/contributing.md
index 9b5851d..07cb106 100644
--- a/docs/contributing.md
+++ b/docs/contributing.md
@@ -12,7 +12,9 @@ disabled. Reference the JIRA key in your pull request title, 
e.g.
 
 ## Building and testing
 
-Requires a Rust toolchain and `protobuf-compiler`.
+Requires a Rust toolchain and `protobuf-compiler`. Non-Windows aarch64 builds
+also need the target C tools described in the
+[source build requirements](installation.md#build-from-source).
 
 ```bash
 cargo build                 # build the library crates
diff --git a/docs/installation.md b/docs/installation.md
index 311ed30..b7aaf9a 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -38,6 +38,11 @@ cargo build --release
 **Build requirements:**
 - Rust toolchain (`rustup` / `cargo`)
 - `protobuf-compiler`
+- For non-Windows aarch64 targets, a C compiler/assembler and archiver that
+  support the target, required by `sha2-asm`. Install `build-essential` on
+  Debian/Ubuntu or Xcode Command Line Tools on macOS. Cross-compilation needs
+  the corresponding target tools, configured for the `cc` crate (for example,
+  through its target-specific `CC_<target>` and `AR_<target>` variables).
 
 ## Python (Drop-in Replacement)
 
@@ -88,6 +93,10 @@ pip install dist/pyspark_client_rust-*.whl
 - Python 3.9 or later
 - Rust toolchain (`rustup` / `cargo`)
 - `protobuf-compiler`
+- For non-Windows aarch64 targets, the C toolchain described in the
+  [Rust source build requirements](#build-from-source).
+
+Installing a prebuilt Python wheel does not require a C toolchain.
 
 ## Version and Compatibility
 


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to