[
https://issues.apache.org/jira/browse/FLINK-38562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18132196#comment-18132196
]
Sangkyoon Nam commented on FLINK-38562:
---------------------------------------
[~dianfu], I'd do FLINK-38562 as one PR and would like your view on two scope
choices first.
The plan is an {{EmbeddedPythonAsyncWaitOperator}} that ports
{{AsyncWaitOperator}}'s control logic onto the Java ordered and unordered
queues, with Python running {{async_invoke}} on an asyncio thread and Java
completing queue entries through the mailbox. A local prototype on a patched
2.4-SNAPSHOT build passed the eight existing non-retry {{AsyncFunctionTests}}
in thread mode. Recovery, {{HEAD}} chaining, and the proposed shutdown and
failure handling aren't tested yet; a build against 2.5-SNAPSHOT is still
needed.
# {{ChainingStrategy.HEAD}} in the first version, and retry as a separate
sub-task like the process-mode split. With default chaining the operator was
chained into a legacy source thread and {{yield()}} failed with "Illegal thread
detected". Until retry lands, thread mode would reject retry strategies.
# Fixing an existing embedded converter bug first in its own ticket: a {{map}}
returning {{Row([Row(1), Row(2), Row(3)])}} with an {{OBJECT_ARRAY}} field of
rows gives {{[3, 3, 3]}} in thread mode instead of {{[1, 2, 3]}}. Async results
go through the same converters.
Do these two work for you?
> Support thread mode of async function in Python DataStream API
> ---------------------------------------------------------------
>
> Key: FLINK-38562
> URL: https://issues.apache.org/jira/browse/FLINK-38562
> Project: Flink
> Issue Type: Sub-task
> Components: API / Python
> Reporter: Dian Fu
> Assignee: Sangkyoon Nam
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
