The GitHub Actions job "Build and push images" on texera.git/main has failed. Run started by GitHub user xuang7 (triggered by xuang7).
Head commit for run: e793be502c050a9be05bc55d67bdad8eaf0ae418 / Eugene Gu <[email protected]> fix(pyamber): coerce numpy scalar types (np.integer / np.bool_) in cast_to_schema (#6278) ### What changes were proposed in this PR? Pandas-based Python operators naturally produce **numpy scalar types** at the `Tuple` output boundary, and the Python worker rejects them: - `df["x"].sum()` / `.max()` / `.count()` return `numpy.int64` - `df["x"].any()` or any numpy comparison returns `numpy.bool_` On output, `Tuple.finalize(schema)` runs `cast_to_schema()` then a strict `isinstance(value, expected_python_type)` check in `validate_schema()`. Because numpy's integer and bool scalar types are **not** subclasses of Python `int` / `bool` (`isinstance(np.int64(5), int)` and `isinstance(np.bool_(True), bool)` are both `False`), the check fails and the operator crashes with `TypeError: Unmatched type for field ...`, even though the value is semantically correct (`np.int64(5)` is exactly the integer 5; the field is declared INTEGER). This is a direct follow-up to #6053, which added `float → int` coercion for INT/LONG fields. That fix only handled `np.float64` (which happens to subclass Python `float`); numpy integer and bool scalars were still rejected. This PR extends the same mechanism in `cast_to_schema()` to cover them: - **INT/LONG fields:** accept `numpy.integer` scalars by normalizing them (and integral floats) into a candidate `int`, then applying the existing `INTEGRAL_TYPE_RANGES` bounds check once. `int()` is exact for numpy integers, so there is no precision loss. Out-of-range values are left unchanged so `validate_schema()` still fails loudly (no int32 overflow / silent corruption). - **BOOL fields:** convert `numpy.bool_` to Python `bool` in a separate branch gated on the BOOL target type, so `bool` and `int` never cross-coerce (`np.bool_` is not `numpy.integer`, and a plain Python `bool` — an `int` subclass — is never coerced into an integer field). `validate_schema()` is unchanged; it remains the strict backstop, so any type this PR does not explicitly coerce still fails loudly rather than passing silently. **Out of scope:** `np.float32` is not a Python `float` subclass, so an integral `np.float32` into an INT field is still rejected. It is intentionally left out of this PR to keep the change focused; it can be folded into the same float-normalization path in a follow-up if desired. ### Any related issues, documentation, discussions? - Fixes #6277 - Follow-up to #6053 (integral `float → int` coercion for INT/LONG fields). ### How was this PR tested? Added unit tests to `amber/src/test/python/core/models/test_tuple.py`. Results: ``` # Python unit tests pytest src/test/python/core/models/test_tuple.py -q -> 84 passed ruff check / ruff format --check -> clean # Backend suite sbt scalafmtCheckAll -> exit 0 sbt "scalafixAll --check" -> exit 0 sbt test -> 1717 passed, 0 failed ``` Adversarial check: reverting `cast_to_schema()` to the pre-fix version turns exactly the coercion cases red while the guard cases stay green; restoring the fix returns to all-green. Every pass-case asserts the concrete Python type, so a test cannot pass without the fix. Also reproduced end-to-end in the platform: `CSV File Scan` → `Python UDF` (`yield {"total_age": table["age"].sum()}` / `{"has_senior": (table["age"] > 60).any()}`) crashed with `Unmatched type ... numpy.int64` / `numpy.bool` before the fix and runs correctly after. ### Was this PR authored or co-authored using generative AI tooling? Co-authored by: Claude Code (Claude Fable 5) --------- Signed-off-by: Eugene Gu <[email protected]> Co-authored-by: Copilot Autofix powered by AI <[email protected]> Report URL: https://github.com/apache/texera/actions/runs/29366364664 With regards, GitHub Actions via GitBox
