kaxil commented on code in PR #74662:
URL: https://github.com/apache/airflow/pull/74662#discussion_r4268337613
##########
providers/common/ai/tests/unit/common/ai/batch/test_anthropic.py:
##########
@@ -266,6 +267,59 @@ def test_over_limit_request_count_rejected(self, adapter,
str_spec):
)
+def _inline_payload_bytes(client) -> int:
+ """
+ Return the bytes the SDK's own encoder produces for the built requests.
+
+ Deriving the expectation from the encoder rather than repeating
``_utf8_size``'s
+ expression keeps it independent of the code under test, and turns these
tests into
+ a tripwire for an SDK that changes how it serializes a request. The
per-request sum
+ ignores the constant ``{"requests": [...]}`` framing, as
``validate_requests`` does.
+ """
+ built = client.messages.batches.created[0]["requests"]
+ return sum(len(openapi_dumps(item)) for item in built)
+
+
+class TestPayloadSizeEstimate:
+ @pytest.mark.parametrize(
+ "prompt",
+ [
+ pytest.param("abc", id="ascii"),
+ pytest.param("δΈζε", id="cjk"),
+ pytest.param("πππ", id="non-bmp"),
+ ],
+ )
+ def test_estimate_matches_the_inline_payload_bytes(self, adapter, client,
str_spec, prompt):
+ requests = [{"prompt": prompt}]
Review Comment:
With a single request this still passes if the loop did `total_bytes =
_utf8_size(item)` instead of summing. Two prompts of different lengths, like
the OpenAI size test, would cover the running total. If the framing gets
counted, the expectation can become `len(openapi_dumps({"requests": built}))`,
and the helper docstring's "constant" framing goes away (the commas grow with
the request count).
##########
providers/common/ai/src/airflow/providers/common/ai/batch/anthropic.py:
##########
@@ -227,7 +241,7 @@ def validate_requests(
)
total_bytes = 0
for index, item in enumerate(built):
- total_bytes += len(json.dumps(item))
+ total_bytes += _utf8_size(item)
Review Comment:
Now that each request is measured exactly, the part that isn't counted
matters. The body the SDK sends is `{"requests":[...]}`, so it's this sum plus
15 bytes of framing plus one comma between each pair of requests (two small
requests: 88 byte body vs a 72 byte sum). At 100,000 requests that's about 100
KB, so if the API counts its limit against the request body, a batch in that
window passes here and gets refused there. Before this change the spaces from
the default separators over-counted by far more than that, which hid it.
Starting `total_bytes` at `len('{"requests":[]}')` and adding 1 for every
request after the first would make it exact, the same way `openai.py` adds 1
per JSONL newline. The docstring on lines 84-85 would need the matching update
(and the default separators add one byte each, not two).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
