PG1204 opened a new pull request, #7243:
URL: https://github.com/apache/texera/pull/7243
### What changes were proposed in this PR?
In the HuggingFace operator's `_read_binary_value`, the final fallback
decoded a column value with `base64.b64decode(val)`. Without `validate=True`,
`b64decode` silently ignores characters outside the base64 alphabet, so
ordinary text "successfully" decodes into garbage bytes and the intended
`val.encode("utf-8")` fallback rarely fires and a plain-text cell could be
turned into garbage and sent as image/audio bytes.
Adding `validate=True` makes `b64decode` reject non-base64 input (spaces,
punctuation, newlines), so real text now correctly falls through to UTF-8. Only
this fallback call changes; the `data:`-URL decodes are left as-is (genuine
base64 that can legitimately contain whitespace).
### Any related issues, documentation, discussions?
Closes #7200.
### How was this PR tested?
`sbt "WorkflowOperator/testOnly
org.apache.texera.amber.operator.huggingFace.*
org.apache.texera.amber.util.PythonCodeRawInvalidTextSpec"`: passes (125
tests). Added a test asserting the fallback uses `validate=True`;
`PythonCodeRawInvalidTextSpec` py-compiles the generated Python. scalafmt clean.
### Was this PR authored or co-authored using generative AI tooling?
This PP was co-authored with Claude Opus 4.8 in compliance with ASF policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
