sshiv012 commented on code in PR #8769:
URL: https://github.com/apache/texera/pull/8769#discussion_r4201345992
##########
amber/src/main/python/core/runnables/data_processor.py:
##########
@@ -130,11 +137,26 @@ def _set_output_tuple(self, output_iterator:
Iterator[Optional[TupleLike]]) -> N
"""
for output in output_iterator:
# output could be a None, a TupleLike, or a TableLike.
+ if isinstance(output, dict) and any(
Review Comment:
Good catch, and you're right. On main that row just gets pickled into the
BINARY column, and I'd turned it into an error. I moved the dict check so it
only runs after `finalize` actually fails, catching `TypeError` as well as
`KeyError`. Valid rows don't get checked at all now. I added a test for the
BINARY case so this doesn't come back.
##########
amber/src/main/python/core/runnables/data_processor.py:
##########
@@ -94,7 +97,11 @@ def process_tuple(self) -> None:
while not finished_current.is_set():
with self._executor_session() as (executor, port_id):
tuple_ =
self._context.tuple_processing_manager.get_input_tuple()
- self._set_output_tuple(executor.process_tuple(tuple_, port_id))
+ self._set_output_tuple(
+ require_yielded(
Review Comment:
Yep, missed that one. I only wrapped the call sites I thought of, not every
place that calls into the user's code. `on_finish` goes through the same check
now, and there's a test for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
