This is an automated email from the ASF dual-hosted git repository.

dbtsai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark-connect-rust.git


The following commit(s) were added to refs/heads/master by this push:
     new b6d340c  [SPARK-60942][CONNECT] Keep artifact limit docs and 
diagnostics in sync
b6d340c is described below

commit b6d340c42272fc13c35f130073b4e17b7a8446e1
Author: DB Tsai <[email protected]>
AuthorDate: Fri Nov 6 22:07:28 2026 -0800

    [SPARK-60942][CONNECT] Keep artifact limit docs and diagnostics in sync
    
    ### What changes were proposed in this pull request?
    
    Follow up on #166 by referencing the hashing byte/blob limit constants in 
the helper documentation and interpolating the file-upload fixture's 
`chunk_size` in its failure diagnostic.
    
    ### Why are the changes needed?
    
    Future limit or fixture-size edits should not leave duplicated numbers in 
the documentation or diagnostic. This addresses two minor findings from the 
post-merge review of SPARK-60942.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    - `git diff --check` and Python syntax parsing passed.
    - Python AST comparison confirms the assertion message is the only 
executable-source change; Rust source is identical after excluding 
documentation comments.
    - Evaluated the diagnostic with 32 KiB and 64 KiB fixture values; it 
reports the corresponding byte count.
    - Genuine local Claude Code reviewed the complete follow-up patch, verified 
both fixes, and found no correctness or blocking issues. Its two optional 
wording nits were assessed and retained as nonblocking tradeoffs. The patch was 
reviewed again after integration with current master, with no new findings.
    
    Local Cargo/rustdoc, native-extension tests and separate API introspection 
could not run because this environment lacks a usable Rust toolchain and 
private extension/dependencies. The local Cargo attempts failed before Cargo 
launched. GitHub CI supplies the Rust, coverage and official Connect/pandas 
parity results. Its advisory workspace Clippy step did not complete because the 
lint runner lacks the wasm32-unknown-unknown standard library; that build 
configuration predates this cleanup.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: OpenAI Codex (GPT-6); reviewed with Claude Code 2.1.314 and 
2.1.319 using configured model `system.ai.claude-fable-5-1`.
    
    This pull request and its description were written by Isaac.
    
    Closes #287 from dbtsai/codex/postmerge-pr-166-review-cleanups.
    
    Authored-by: DB Tsai <[email protected]>
    Signed-off-by: DB Tsai <[email protected]>
    Co-authored-by: Isaac <[email protected]>
---
 crates/spark-connect-core/src/client.rs | 3 ++-
 python/tests/test_arrow_handoff.py      | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/crates/spark-connect-core/src/client.rs 
b/crates/spark-connect-core/src/client.rs
index 62c1652..0623378 100644
--- a/crates/spark-connect-core/src/client.rs
+++ b/crates/spark-connect-core/src/client.rs
@@ -60,7 +60,8 @@ const ARTIFACT_HASH_INLINE_MAX_BYTES: usize = 256 * 1024;
 /// Bound per-blob digest and encoding work even when every blob is empty.
 const ARTIFACT_HASH_INLINE_MAX_BLOBS: usize = 128;
 
-/// Hash up to 128 blobs totaling at most 256 KiB inline; offload larger 
batches.
+/// Hash batches inline up to [`ARTIFACT_HASH_INLINE_MAX_BLOBS`] blobs and
+/// [`ARTIFACT_HASH_INLINE_MAX_BYTES`] total bytes; offload larger batches.
 /// Returns the original buffers for upload framing. If the caller is dropped 
after
 /// blocking hashing starts, the task retains the blob buffers until it 
finishes.
 async fn hash_artifact_blobs(blobs: Vec<Vec<u8>>) -> Result<(Vec<Vec<u8>>, 
Vec<String>)> {
diff --git a/python/tests/test_arrow_handoff.py 
b/python/tests/test_arrow_handoff.py
index b40a434..af993c4 100644
--- a/python/tests/test_arrow_handoff.py
+++ b/python/tests/test_arrow_handoff.py
@@ -760,7 +760,8 @@ def test_file_upload_attribution_on_every_frame(
         payloads = {context["payload"] for context in state["upload_contexts"]}
         assert payloads == {"batch", "begin_chunk", "chunk"}, (
             f"Expected all artifact frame kinds, got {payloads}. "
-            "This fixture assumes a 32 KiB CHUNK_SIZE; adjust its sizes if 
CHUNK_SIZE changes."
+            f"This fixture assumes a {chunk_size}-byte CHUNK_SIZE; "
+            "adjust its sizes if CHUNK_SIZE changes."
         )
         assert state["cache"][artifact_names[0]] == small.read_bytes()
         assert state["cache"][artifact_names[1]] == large.read_bytes()


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to