JingsongLi commented on PR #10179:
URL: https://github.com/apache/paimon/pull/10179#issuecomment-5951761078
Reviewed head `d15e9354dd3cc2f3e55a71b42c09b0215de49d52`. The REST
authorization use case has end-to-end value, but I found one cross-engine
authorization contract gap.
**[P2] Align OVERLAY arithmetic at INT boundaries before accepting these
authorization rules** (`predicate_json_parser.py:506-507`). Both operands are
accepted Java INT values, but `pos - 1` and `pos + replaced` use unbounded
Python arithmetic here, while `OverlayTransform.transform` evaluates those
expressions as signed 32-bit Java arithmetic. For the same valid transform
`{"name":"OVERLAY","inputs":[{"index":1,"name":"s","type":"STRING"},"x",1,2147483647]}`
and `s = "hello"`, the actual Java transform returns `"xhell"`, whereas this
Python implementation returns `"x"`. A row-filter `EQUAL "x"` therefore admits
the row in PyPaimon and rejects it in Java. I reproduced the Python result
through an actual Parquet table write, commit, reload, auth scan and read,
rather than only calling the helper.
The existing Java overflow behavior may itself deserve correction; this is
not a request to preserve it as the intended SQL behavior. Please establish the
arithmetic contract in both implementations (or reject unsupported boundary
combinations consistently), and add differential filter/mask cases before
treating the new Python transforms as equivalent consumers of Java-authored
auth rules.
Validation: 330 Python parser/auth tests passed; 11 Java reference tests
passed with normal Maven checks. Ten actual persisted-table controls passed for
all five transforms, including projected-column widening, Unicode and null
handling, and each new transform is rejected by the baseline implementation. An
independent comparison of 2,544 valid transform cases against the compiled Java
classes found 276 mismatches, all in OVERLAY overflow cases; ordinary cases and
the other four transforms agreed. Configured flake8 and Python 3.6 grammar
checks passed; current-head CI is green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
