kz930 opened a new pull request, #7569: URL: https://github.com/apache/texera/pull/7569
### What changes were proposed in this PR? Sentiment Analysis, Spam SMS Detection and Text Summarization each passed their text column straight to a tokenizer or pipeline. An empty cell arrives as `None`, which transformers rejects with `ValueError: You need to specify either text or text_target` for the first two and `ValueError: text input must be of type str ...` for the third, ending the run. An empty value is ordinary input here. A blank CSV cell arrives as null, since univocity returns null for an empty field and `AttributeTypeUtils.parseField` passes it through by design. All three now yield the row with their result attributes left empty. That follows what the Hugging Face inference operator does with a row it cannot process: `HuggingFaceCodegenBase` appends the error to that row's results and continues rather than dropping it. The visualization operators drop such rows instead, with `dropna(subset=[...])`, but their output is a chart, where a missing row costs nothing. These three add columns to each input row, so dropping would take the user's row out of the output along with the value the model had nothing to say about. Whitespace-only text takes the same path, for the same reason. ### Any related issues, documentation, discussions? Closes #7549 ### How was this PR tested? Each of the three specs gains a case asserting the generated Python guards the empty cell before the value reaches the tokenizer or pipeline. All three fail on the previous behavior, 79 passed / 3 failed before the change and 82 / 0 after. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
